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FOREWORD 


The  high  cost  of  training  on  operational  equipment,  safety 
and  environmental  considerations,  and  lack  of  adequate  resources 
have  increased  the  need  for  effective  training  devices  and  simu¬ 
lations.  Progress  in  instructional  technology,  such  as  improved 
simulation  technology  and  computer-assisted  instruction,  has 
greatly  expanded  the  range  of  options  available  to  the  training 
device  designer.  This  expansion  has  produced  an  increase  in  the 
complexity  of  the  process  for  designing  training  devices. 

The  Project  Manager  for  Training  Devices  (PM  TRADE)  and  the 
U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sci¬ 
ences  (ARI) /PM  TRADE  Field  Unit  are  collocated  to  encourage  the 
integration  of  complementary  engineering  and  behavioral  efforts 
in  fielding  training  devices  and  simulators.  This  report  docu¬ 
ments  an  ARI  survey  of  PM  TRADE  engineers  who  design  training 
devices.  The  survey  focused  on  members  of  the  Research  and 
Engineering  Management  Branch  (E  Division)  of  PM  TRADE.  The 
objectives  of  the  research  were  to  (1)  describe  a  baseline  pro¬ 
cess  model  of  the  role  of  engineers  in  the  concept  formulation 
process  (CFP)  for  training  devices,  (2)  describe  the  need  for 
training  decision  aids  within  E  Division,  (3)  describe  the  need 
for  improving  the  information  flow  within  PM  TRADE  and  between  PM 
TRADE  and  other  organizations  involved  in  developing  training 
devices,  (4)  capture  engineers'  ideas  for  improving  E  Division 
operations,  and  (5)  compare  the  fit  between  the  baseline  process 
model  and  a  prototype  decision  support  aid  for  device  design. 

This  research  was  performed  as  part  of  the  program  task 
"Advanced  Technology  for  the  Design  of  Training  Devices."  The 
sponsor  of  the  research  was  PM  TRADE.  The  research  was  accom¬ 
plished  under  the  Memorandum  of  Understanding  between  PM  TRADE 
and  ARI  dated  14  July  1986.  Preliminary  research  findings  were 
briefed  on  11  December  1989  to  Dr.  Ronald  Hofer,  Chief,  Research 
and  Management  Division,  PM  TRADE.  Dr.  Hofer  reviewed  the  final 
report . 
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FOREWORD 


The  high  cost  of  training  on  operational  equipment,  safety 
and  environmental  considerations,  and  lack  of  adequate  resources 
have  increased  the  need  for  effective  training  devices  and  simu¬ 
lations.  Progress  in  instructional  technology,  such  as  improved 
simulation  technology  and  computer-assisted  instruction,  has 
greatly  expanded  the  range  of  options  available  to  the  training 
device  designer.  This  expansion  has  produced  an  increase  in  the 
complexity  of  the  process  for  designing  training  devices. 

The  Project  Manager  for  Training  Devices  (PM  TRADE)  and  the 
U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sci¬ 
ences  (ARI)/PM  TRADE  Field  Unit  are  collocated  to  encourage  the 
integration  of  complementary  engineering  and  behavioral  efforts 
in  fielding  training  devices  and  simulators.  This  report  docu¬ 
ments  an  ARI  survey  of  PM  TRADE  engineers  who  design  training 
devices.  The  survey  focused  on  members  of  the  Research  and 
Engineering  Management  Branch  (E  Division)  of  PM  TRADE.  The 
objectives  of  the  research  were  to  (1)  describe  a  baseline  pro¬ 
cess  model  of  the  role  of  engineers  in  the  concept  formulation 
process  (CFP)  for  training  devices,  (2)  describe  the  need  for 
training  decision  aids  within  E  Division,  (3)  describe  the  need 
for  improving  the  information  flow  within  PM  TRADE  and  between  PM 
TRADE  and  other  organizations  involved  in  developing  training 
devices,  (4)  capture  engineers’  ideas  for  improving  E  Division 
operations,  and  (5)  compare  the  fit  between  the  baseline  process 
model  and  a  prototype  decision  support  aid  for  device  design. 

This  research  was  performed  as  part  of  the  program  task 
"Advanced  Technology  for  the  Design  of  Training  Devices."  The 
sponsor  of  the  research  was  PM  TRADE.  The  research  was  accom¬ 
plished  under  the  Memorandum  of  Understanding  between  PM  TPiADE 
and  ARI  dated  14  July  1986.  Preliminary  research  findings  were 
briefed  on  11  December  1989  to  Dr.  Ronald  Hofer,  Chief,  Research 
and  Management  Division,  PM  TRADE.  Dr.  Hofer  reviewed  the  final 
report . 
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ENGINEERING  FUNCTIONS  IN  FORMULATING  TRAINING  DEVICE  CONCEPTS 


EXECUTIVE  SUMMARY 


Requirement: 

The  goal  of  this  project  was  to  define  a  baseline  process 
model  of  the  role  of  engineers  in  the  concept  formulation  process 
(CFP)  for  training  devices,  identify  targets  of  opportunity  for 
increasing  the  effectiveness  and  efficiency  of  CFP,  and  recommend 
courses  of  action.  A  major  concern  of  this  project  was  the  exam¬ 
ination  of  learning  principles  considered  by  engineers  in  train¬ 
ing  device  CFP. 


Procedure: 

The  training  device  CFP  was  defined  as  it  is  described  in 
Army  regulations,  SOPs,  and  pamphlets.  A  questionnaire  was 
designed  for  collecting  data  on  the  work  experience  of  engineers, 
job  task  descriptions,  obstacles  to  job  performance,  and  ideas 
for  improving  job  performance.  This  questionnaire  was  adminis¬ 
tered  to  all  engineers  within  PM  TRADE'S  Engineering  and  Research 
Management  Division.  Fifteen  engineers  were  then  selected  for 
interviews  based  on  responses  to  questionnaire  items.  Prelimi¬ 
nary  conclusions  were  discussed  with  various  engineers,  and 
conclusions  were  revised  where  appropriate  to  reflect  engineer 
comments . 


Findings: 

Engineers  make  decisions  about  materiel  solutions  to 
training  requirements  that  require  consideration  of  the  princi¬ 
ples  of  learning,  but  the  principles  considered  vary  among 
engineers.  All  engineers  appear  to  be  concerned  with  identifying 
the  minimum  cue  and  response  fidelity  requirements  necessary  for 
effective  learning.  Some  engineers  go  beyond  the  consideration 
of  fidelity  requirements  to  address  such  issues  as  how  to  measure 
performance  and  provide  feedback. 

The  task  of  obtaining  the  information  about  training 
requirements  needed  to  apply  learning  principles  to  device  design 
often  continues  throughout  the  CFP.  No  standard  set  of  proce¬ 
dures  is  used  for  considering  learning  principles  in  the  design 
of  training  devices. 
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Five  factors  were  identified  that  influence  training  device 

CFP: 

•  whether  a  school  specifies  that  it  wants  a  specific 
training  device; 

•  the  extent  to  which  the  project  involves  novel  applica¬ 
tions  of  training  technology; 

•  whether  the  project  addresses  individual  training  or 
collective  training; 

•  training  device  policy  statements; 

•  whether  the  project  addresses  school  or  unit  training. 

Targets  of  opportunity  for  improving  the  training  device  CFP 
include  linking  learning  variables  to  materiel  variables  as  a 
function  of  project  categories,  and  clarification  of  the  intended 
relationships  between  major  components  of  CFP.  An  example  of  the 
latter  target  is  the  Trade-Off  Analysis  (TOA)  conducted  to  iden¬ 
tify  the  best  materiel  solution  and  the  Cost  and  Training  Effec¬ 
tiveness  Analysis  (CTEA) . 


Utilization  of  Findings: 

These  findings  are  being  used  by  PM  TRADE  to  increase  the 
effectiveness  and  efficiency  of  targeted  areas  of  CFP.  These 
findings  are  also  being  used  to  guide  a  joint  PM  TRADE  and  ARI 
project  to  refine  the  Optimization  of  Simulation-Based  Training 
Systems  (OSBATS)  decision  support  system. 
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ENGINEERING  FUNCTIONS  IN  FORMULATING 
TRAINING  DEVICE  CONCEPTS 

Introduction 

The  Importance  of  the  Training  Device  Concent  Formulation  Process 

The  high  cost  of  training  on  operational  equipment,  combined 
with  safety  considerations  and  lack  of  adequate  resources,  have 
acted  as  stimuli  to  the  development  of  training  devices.  Certain 
training  devices,  such  as  SIMNET  (simulation  networking) ,  provide 
a  means  of  conducting  maneuver  training  without  operational 
equipment  and  help  units  to  compensate  for  reductions  in  OPTEMPO. 
Other  devices,  such  as  the  Multiple  Integrated  Laser  Engagement 
Simulation  System  (MILES) ,  help  to  reduce  ammunition  costs 
associated  with  training  while  also  providing  a  degree  of  realism 
that  is  not  normally  possible  outside  of  a  combat  situation. 
Effective  training  devices  can  lead  to  better,  safer  and  less 
costly  training. 

The  technological  tools  that  can  be  applied  to  training 
device  design  are  continually  growing  in  number  and  power.  The 
ability  to  simulate  the  combat  environment  has  been  enhanced  by 
procedures  for  preparing  high-resolution  graphics,  improved 
visual  and  auditory  signal  generators,  and  increasing  power  of 
microprocessors.  These  innovations  make  it  technically  feasible 
to  create  training  environments  that  were  not  possible  a  few 
years  ago,  but  the  costs  associated  with  applying  these 
innovations  can  be  high. 

Cost-effective  application  of  technology  to  the  design  of 
training  devices  requires  careful  matching  of  training 
requirements  and  training  device  solutions.  The  process  of 
formulating  a  concept  for  a  training  device  is  a  critical  phase 
of  the  device  design  process  in  which  trainers  convey  information 
to  engineers  about  training  device  needs,  and  engineers  begin  to 
define  a  materiel  solution  to  these  needs.  The  goal  of  the 
training  device  concept  formulation  process  (CFP)  is  to  identify 
a  cost-effective  device  or  combination  of  devices  that  might  be 
used  to  address  a  training  requirement. 

This  report  presents  the  results  of  an  effort  to  define 
methods  for  aiding  the  training  device  CFP.  It  emphasizes 
engineers'  consideration  of  factors  influencing  skill  acquisition 
and  retention  when  defining  materiel  solutions  to  training 
requirements . 
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Background;  Training  Device  CFP  as  Described  by  Regulation 

The  CFP  process  for  weapon  systems  and  training  devices  is 
governed  by  Army  Regulations  70-1  (Department  of  the  Army,  1986) 
and  71-9  (Department  of  the  Army,  1987) .  These  regulations  are 
supplemented  with  guidance  in  the  form  of  a  Memorandum  of 
Instruction  (U.S.  Army  Materiel  Command  and  U.S.  Army  Training 
and  Doctrine  Command,  1988)  and  AMC-TRADOC  PAM  70-2  (U.S.  Army 
Materiel  Command  and  U.S.  Army  Training  and  Doctrine  Command, 

1988)  . 

These  regulations  and  the  supplementary  guidance  describe  the 
CFP  from  a  high  level  perspective.  This  high  level  view  of  the 
CFP  as  it  applies  to  training  devices  is  presented  below. 

Before  the  formal  start  of  the  CFP  a  dialogue  is  maintained 
between  the  U.S.  Army  Project  Manager  for  Training  Devices 
(PM  TRADE)  and  school  training  developers  concerning  the  training 
requirements  to  be  addressed  by  a  device  (U.S.  Army  Materiel 
Command  and  U.S.  Army  Training  and  Doctrine  Command,  1988).  The 
purpose  of  the  dialogue  is  to  insure  that  adequate  information  is 
developed  about  a  requirement  to  facilitate  a  search  for  a 
materiel  solution.  PM  TRADE  initiates  the  CFP  when  it  receives  a 
valid  description  of  training  requirements  from  a  school  (PM 
TRADE,  1986). 

PM  TRADE  takes  the  lead,  with  TRADOC  support,  in  performing  a 
Trade-off  Detemmination  (TOD)  to  provide  the  information  needed 
to  compare  device  alternatives  (Department  of  the  Army,  1988) . 
"The  TOD  documentation  includes  technical  analyses  or  trade¬ 
offs;  risks,  capabilities  needed;  costs;  integrated  logistic 
support  (ILS)  requirements;  estimated  total  Army  manpower 
requirements;  health,  safety,  and  human  factors  engineering  (HFE) 
requirements;  and  ecological  factors"  (Army  Training  Support 
Center,  1988)  to  establish  a  set  of  viable  alternatives. 

TRADOC  takes  the  lead,  with  support  from  PM  TRADE,  in 
conducting  a  Trade-off  Analysis  (TOA)  to  identify  the  preferred 
materiel  solution.  To  a  large  extent,  the  conduct  of  the  TOA 
involves  analyzing  the  data  collected  in  the  TOD.  TRADOC  takes 
the  lead  in  preparing  a  Cost  and  Training  Effectiveness  Analysis 
(CTEA)  that  compares  the  cost  and  training  effectiveness  of 
training  device  alternatives  (Department  of  the  Army,  1988) . 

The  TOD,  TOA,  and  CTEA  are  intended  to  be  integrated  with  one 
another,  and  they  are  expected  to  overlap  in  time  (see  Figure  1) . 

PM  TRADE  then  takes  the  lead,  with  support  from  TRADOC,  in 
preparing  the  Best  Technical  Approach  (BTA) .  This  phase  involves 
refining  and  further  specifying  the  description  of  the  selected 
materiel  solution  in  adequate  detail  to  allow  for  writing 
technical  specifications.  In  addition,  engineers  are  responsible 
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Figure  1 .  The  training  device  concept  formulation  process  (CFP) 
as  described  by  regulations. 
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for  preparing  a  Baseline  Cost  Estiirate  (BCE)  to  provide  the 
government  with  a  valid  estimate  of  the  life  cycle  cost  of  the 
training  device. 

CFP  ends  when  the  various  documents  constituting  the  CFP 
have  been  approved  at  the  Milestone  I/II  IPR.  The  concept  for 
the  device  then  enters  configuration  management  control.  That 
is,  the  technical  specifications  may  be  changed  only  if  approved 
by  a  Configuration  Control  Board  internal  to  PM  TRADE  (PM  TRADE, 
1986)  . 

A  different  picture  of  the  training  device  CFP  can  be  gained 
by  viewing  it  as  a  series  of  reviews  in  which  interested  parties 
come  together  to  insure  that  adequate  information  has  been 
developed  to  move  on  to  the  next  phase  of  CFP.  Table  1  is  based 
upon  PM  TRADE  Standard  Operating  Procedure  66  (1986),  and  it 
lists  the  major  reviews  in  CFP  and  describes  the  purpose  of  each 
review. 

Project  Goals  and  Objectives 

Turning  descriptions  of  training  deficiencies  into  hardware 
and  software  specifications  for  training  devices  requires  the 
expertise  of  training  developers  and  materiel  developers:  CFP  is 
a  joint  effort  of  school  training  developers  and  PM  TRADE 
engineers. 

PM  TRADE  has  three  divisions  involved  in  CFP  (PM  TRADE  SOP 
66,  1986) .  Project  management  is  provided  by  the  Resources 
Management  Division  (R  Division) .  The  Technical  Support  and 
Readiness  Division  (S  Division)  is  concerned  with  MANPRINT, 
logistical,  product  assurance  and  RAM  (reliability,  availability 
and  maintainability)  aspects  of  training  device  concepts.  PM 
TRADE'S  Research  and  Engineering  Management  Division,  known  as  E 
Division,  leads  the  system  engineering  activities  during  concept 
formulation  that  assess,  integrate,  and  develop  training  device 
design  concepts  and  cost  estimates.  Key  elements  of  E  Division's 
mission  that  relate  directly  to  concept  formulation  are  to 
maintain  a  technical  database  appropriate  to  the  concept 
formulation  process,  work  with  other  organizations  in  formulating 
training  device  concepts,  and  guide  technological  research  to 
insure  that  emerging  technology  can  be  applied  to  future  concept 
formulation  efforts. 

E  Division  is  divided  into  three  branches.  Two  of  these 
branches.  Engineering  Concepts  and  Engineering  Development,  serve 
identical  functions.  These  branches  are  concerned  with  developing 
the  information  used  to  compare  training  device  options  and 
writing  the  functional  specifications  for  the  training  device 
option  or  options  that  are  selected.  In  terms  of  the  concept 
formulation  process  shown  in  Figure  1,  these  branches  perform  a 
TOD,  support  the  conduct  of  a  TOA,  develop  the  BTA,  and  develop 
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TABLE  1.  PURPOSE  OF  THE  VARIOUS  DECISION  MEETINGS  AND  REVIEWS  IN 
TRAINING  DEVICE  CFP 


REVIEW 

PROJECT 

SCREENING 

COMMITTEE 

(PSC) 

JOINT  WORKING 
GROUP  I  (JWGI) 


MATERIEL 
ACQUISITION 
REVIEW  BOARD 
(MARB)  I 

JWG  2 


MARB  II 


AMC  MILESTONE 
I/II  POSITION 


MILESTONE  I/II 
I  PR 

DATA  REQUIRE¬ 
MENTS  REVIEW 
BOARD  (DRRB) 

QUARTERLY 

REVIEWS 


PURPOSE 

An  internal  PM  TRADE  review  conducted  to 
assess  whether  PM  TRADE  has  the  manpower  to 
devote  to  a  particular  project. 


A  joint  meeting  of  TRADOC  and  PM  TRADE 
to  decide  if  Training  Requirement  is 
adequately  defined  by  a  TRADOC-prepared  Training 
Development  Need  Statement  (TDNS) 


PM  TRADE  internal  review  of  data  developed 
by  PM  TRADE  for  Trade-Off  Determination  (TOD) 
to  insure  that  all  issues  relevant  to  training 
device  selection  have  been  addressed 

A  joint  meeting  between  TRADOC  and  PM  TRADE 
to  perform  a  Trade-Off  Analysis  (TOA) 

PM  TRADE  internal  review  of  the  BTA  and 
validated  Baseline  Cost  Estimate  (BCE)  to 
insure  that  all  items  are  adequate  to  support 
a  Milestone  I/II  IPR 

A  review  to  adopt  the  AMC  position  regarding 
the  Best  Technical  Approach  (BTA)  in  preparation 
for  the  Milestone  I/II  In  Process  Review  (IPR) 

A  review  to  decide  whether  to  proceed  with  full 
scale  development  of  a  training  device 

A  review  to  validate  the  contents  of  training 
device  procurement  packages 


Quarterly  PM  TRADE  internal  review  of  each 
project  regarding  cost,  schedule  and  performance 
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the  BCE.  The  third  branch,  Technology  Management,  plays  a 
distinctly  different  role  in  the  design  of  training  devices.  ET 
branch  is  concerned  with  guiding  industry  in  the  conduct  of 
training  technology  research  and  development  to  insure  that  the 
outcomes  of  these  efforts  will  be  responsive  to  the  needs  of  the 
Army. 

We  analyzed  E  Division's  functions  in  training  device  CFP  in 
order  to  address  two  goals.  The  first  goal  of  this  effort  was  to 
describe  targets  of  opportunity  for  increasing  the  effectiveness 
and  efficiency  of  CFP.  The  second  goal  was  to  develop 
information  to  guide  the  refinement  of  the  prototype  Optimization 
of  Simulation-Based  Training  Systems  (OSBATS) ,  a  software  package 
and  underlying  model  designed  to  support  the  TOA  portion  of 
concept  formulation  (Sticha,  1990) .  OSBATS  is  a  computer-based 
model  that  interactively  uses  rule-based  procedures  and  databases 
to  aid  in  specifying  the  design  and  use  of  training  equipment  to 
satisfy  training  requirements  at  a  minimum  cost,  or  to  provide 
the  greatest  training  effectiveness  for  a  given  cost  (Singer  and 
Sticha,  1987) .  A  thorough  description  of  the  OSBATS  model  is 
provided  in  Sticha,  Blacksten,  Buede,  Singer,  Gilligan,  Mumaw, 
and  Morrison,  1990) . 

The  objectives  of  this  project  were  to; 

a.  describe  a  baseline  process  model  of  CFP  from  the 
perspective  of  E  Division  engineers; 

b.  describe  the  need  for  training  decision  aids  within  E 
Division; 

c.  describe  the  need  for  improving  the  information 
flow  within  PM  TRADE  and  between  PM  TRADE  and  other 
organizations  involved  in  developing  training  devices; 

d.  capture  engineers'  ideas  for  improving  E  Division 
operations ; 

e.  compare  the  baseline  process  model  of  the  CFP  with  the 
OSBATS  model. 
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Method 


Respondents 

A  questionnaire  was  distributed  to  the  Division  Chief  and  all 
engineers  within  E  Division's  Engineering  Development  (ED) , 
Engineering  Concepts  (EC)  and  Technology  Management  (ET) 

Branches.  ED  and  EC  branches  are  responsible  for  concept 
formulation,  while  ET  branch  is  responsible  for  managing  the 
technology  database.  Nineteen  of  the  21  engineers  working  in  the 
concept  formulation  process  completed  the  questionnaire,  and  four 
of  the  five  engineers  working  in  ET  Branch  completed  the 
questionnaire.  Eleven  engineers  from  ED  or  EC  Branch  were 
interviewed  on  the  concept  formulation  process,  and  two  members 
of  ET  Branch  were  interviewed  regarding  management  of  the 
technology  database. 

Procedure 


Data  were  collected  through  a  review  of  written  CFP  guidance, 
questionnaire  administration,  and  interviews  with  selected 
engineers.  Engineers  were  given  three  weeks  to  complete  the 
questionnaire.  Responses  to  questionnaire  items  were  used  in 
selecting  engineers  to  be  interviewed  and  deciding  which  topics 
to  address  in  each  interview. 

Responses  to  the  background  section  of  the  questionnaire 
provided  information  about  who  had  experience  with  specific 
concept  formulation  tasks,  and  engineers  were  selected  for 
interviews  when  task  requirements  were  found  to  be  unclear. 
Engineers  were  selected  for  interviews,  in  part,  on  the  basis  of 
reported  problems  in  making  training-relevant  decisions  and 
reported  obstacles  to  job  performance. 

The  preliminary  results  and  conclusions  of  questionnaire 
response  analysis  and  interviews  were  discussed  with  the  Division 
Chief  and  two  Branch  Chiefs.  As  a  result  of  these  discussions, 
additional  information  was  obtained  regarding  the  reasons  behind 
selected  observations. 

Questionnaire 

Appendix  A  provides  the  four-section  questionnaire  used  in 
this  project.  The  first  section  was  concerned  with  the  education 
and  experience  of  respondents.  Aspects  of  experience  addressed 
in  this  section  are  listed  below,  and  summaries  of  the  responses 
to  these  questions  are  provided  in  Appendix  B. 

•  Length  of  time  engineer  has  worked  within  E  Division  and 
PM  TRADE  in  general 
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•  Number  of  years  of  experience  designing  training  devices 

•  Types  of  devices  on  which  engineer  has  worked 

•  Identification  of  TRADOC  schools  that  engineer  has  worked 
with  in  the  preceding  two  year  period 

•  Identification  of  formal  reviews  for  which  engineer  has 
developed  input 

•  Experience  or  lack  of  experience  with  the  various 
activities  and  documents  that  might  be  involved  in 
a  concept  formulation  project 

•  The  amount  and  kind  of  training  each  engineer  has  had  in 
systems  engineering 

•  Whether  an  engineer  is  required  to  make  specified 
training  decisions,  if  problems  have  been  encountered  in 
making  these  decisions  because  of  a  lack  of  information 
or  decision  criteria,  and  if  the  engineer  was  able  to 
overcome  the  problem  or  has  knowledge  about  the  problem 

•  The  software  and  databases  used  by  engineers  in  their 
work 

The  second  section  of  the  questionnaire  was  used  to  collect 
data  regarding  the  work  flow  in  CFP.  Engineers  were  asked  to 
list  their  major  job  functions,  list  the  major  tasks  performed 
within  each  function,  describe  the  input  and  output  associated 
with  each  function,  and  list  sources  of  information  used  in 
performing  each  function. 

The  third  section  was  used  to  collect  information  on  the 
decisions  engineers  make  that  require  consideration  of  variables 
influencing  skill  acquisition  and  retention.  Engineers  were 
asked  what  training  or  learning  issues  they  consider  in  their 
work,  and  they  were  asked  to  identify  the  most  significant 
training  decisions  that  bear  upon  the  selection  of  a  Best 
Technical  Approach  (BTA)  to  a  training  device  requirement.  In 
addition,  they  were  asked  to  list  the  most  significant  steps  or 
decision  points  in  arriving  at  the  BTA. 

The  fourth  section  addressed  job  satisfaction  and  defining 
areas  for  improvement  in  concept  formulation.  These  questions 
were: 

•  What  are  the  major  obstacles  to  performing  your  job 
efficiently? 

•  What  information  would  better  assist  you  in  performing 
your  job  duties? 
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•  Of  the  tasks  you  perform  in  your  job,  which  would  you 
most  like  for  someone  else  to  perform?  Why? 

•  Of  the  decisions  that  you  make  in  your  work,  which  would 
you  most  like  to  have  someone  else  make? 

•  Which  parts  of  your  job  give  you  the  greatest 
satisfaction? 

•  What  training  would  you  like  to  receive  that  you  feel 
would  help  you  to  better  perform  your  duties? 
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Baseline  Concept  Formulation  Process  (CFP)  for  Training  Devices 


The  training  device  CFP  is  described  at  a  general  level  in 
regulations  and  guidance  documents.  The  description  of  CFP 
described  here  is  based  upon  responses  to  survey  items,  formal 
interviews,  informal  discussions  with  engineers,  and  observation 
of  engineer's  activities  in  the  context  of  project  meetings.  In 
the  resulting  summary  we  have  tried  to: 

•  identify  and  explain  the  manner  in  which  general 
regulations  are  applied  in  practice 

•  examine  the  depth  of  activities  performed  by  engineers 
in  adequate  detail  to  identify  major  variables  that 
influence  their  jobs 

•  identify  issues  about  the  intent  of  CFP  regulations  that 
have  not  been  resolved  through  application  of  the 
regulations 

•  identify  sources  of  information  currently  used  in 
performing  CFP 

How  CFP  Regulations  and  Guidance  Documents  are  Applied  in 
Practice 


Many  of  the  regulations  and  guidance  documents  for  performing 
CFP  are  written  at  a  general  level  and  apply  to  both  the  prime 
weapon  system  and  training  device  CFP.  This  fact  was  reported  by 
a  number  of  engineers.  However,  even  when  examining  the  training 
device  CFP  at  a  general  level  there  are  important  differences 
between  regulations  or  guidance  and  what  actually  happens  in 
practice  that  help  to  define  targets  of  opportunity  for  improving 
CFP.  Listed  below  are  three  important  ways,  relating  to  school 
description  training  requirements,  in  which  CFP  as  defined  by 
formal  guidance  differs  from  CFP  as  practiced  within  E  Division  . 
For  a  more  detailed  listing  of  CFP  tasks  and  subtasks  performed 
by  engineers  the  reader  is  referred  to  Appendix  C. 

•  E  Division  often  agrees  to  initiate  CFP  before  TRADOC 
has  defined  training  requirements  in  the  manner  specified 
by  regulations  and  guidance  documents.  In  practice,  the 
definition  of  training  requirements  may  continue 
throughout  training  device  CFP,  rather  than  being 
completed  prior  to  the  start  of  CFP. 

•  PM  TRADE  plays  active  roles  in  helping  to  define  the 
training  requirements  to  be  addressed  by  devices  through 
reviews  of  Training  Device  Requirements  (TDRs) ,  Training 
Development  Need  Statements  (TDNS) ,  and  other  documents 
TRADOC  is  responsible  for  preparing. 
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•  In  certain  cases  the  engineer  also  assumes  responsibility 
for  ensuring  that  adequate  task  analyses  and  media 
analyses  are  performed  to  provide  the  information 
required  to  execute  a  TOD. 

The  substantial  role  played  by  engineers  in  attempting  to 
define  the  training  requirements  to  be  addressed  by  a  materiel 
solution  can  be  credited,  in  part,  to  the  fact  that  schools  do 
not  provide  the  descriptions  of  training  deficiencies  as 
specified  in  regulations  and  guidance  documents.  This  is  a 
problem  that  has  previously  been  reported  (Heeringa,  Baum, 

Holman,  and  Peio,  1982) ,  and  it  will  be  discussed  in  greater 
detail  later  in  this  report.  For  the  present  it  is  important  to 
note  that  the  major  deficiencies  in  the  training  requirements 
developed  by  schools  are  a  failure  to  list  the  tasks  to  be 
trained,  and  a  failure  to  describe  the  minimum  fidelity 
requirements  necessary  for  effective  training. 

Would  the  need  for  engineers  to  play  an  active  and 
substantial  role  in  defining  training  requirements  be  reduced  if 
schools  defined  training  requirements  in  accordance  with 
regulations  and  guidance?  Probably  not.  The  reason  for  this 
assessment  is  that  the  guidance  for  describing  training 
requirements  (Army  Training  Support  Center,  1988)  is  essentially 
the  same  for  all  projects.  The  scope  of  current  CFP  projects,  on 
the  other  hand,  ranges  from  part  task  trainers  intended  to  train 
selected  individual  operator  or  maintenance  tasks  for  a  single 
duty  position  to  networked  simulations  at  the  Battalion  Task 
Force  level.  Further,  many  projects  involve  novel  applications 
of  training  technology;  engineers  discover  many  of  their 
information  needs  regarding  training  requirements  during  the 
course  of  the  CFP. 

A  fourth  difference  between  regulations  or  guidance  and 
practice  is  that  engineers  sometimes  perform  a  preliminary  TOA 
for  review  by  schools,  rather  than  waiting  for  schools  to 
initiate  this  process.  This  preliminary  TOA  serves  as  an 
internal  check  on  the  quality  of  the  TODs  performed  by  engineers, 
since  the  TOA  employs  the  information  developed  during  the  TOD. 

Major  Project  Variables  That  Influence  the  Training  Device  CFP 

At  the  task  level,  described  in  Appendix  C,  there  is  a 
roughly  uniform  CFP  within  PM  TRADE  that  differs  from  regulations 
and  other  guidance  as  described  immediately  above.  When  one 
starts  to  look  at  the  subtask  level  and  observes  what  engineers 
do  in  performing  these  subtasks,  it  becomes  apparent  that  the  CFP 
differs  among  projects.  At  the  start  of  data  collection,  we 
expected  differences  in  domain  (e.g.,  armor,  aviation,  combat 
service  support)  to  have  a  significant  effect  on  the  procedures 
employed  during  the  CFP.  Instead,  we  found  that  other  factors 
appeared  to  have  a  more  pronounced  effect  on  the  CFP.  In  all  but 
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one  case,  these  factors  surfaced  during  the  course  of  interviews 
and  in  the  context  of  project  me^^tings  in  which  we  participated. 
These  factors  surfaced  as  problems  in  the  context  of  more  than 
one  project,  while  not  being  common  to  all  projects. 

Five  factors  were  identified  as  having  a  substantial  impact 
on  the  CFP.  These  factors  are: 

•  whether  a  school  specifies  that  it  wants  a  specific 
training  device; 

•  whether  the  project  addresses  individual  training  or 
collective  training; 

•  the  extent  to  which  the  project  involves  novel 
applications  of  training  technology; 

•  training  device  policy  statements; 

•  whether  the  project  addresses  school  or  unit  training. 

Whether  a  TRADOC  School  Specifies  That  It  Wants  a  Specific 
Training  Device 

Requirements  documents  vary  in  terms  of  whether  they  describe 
a  training  requirement  or  specify  a  particular  device. 

A  critical  portion  of  a  school’s  job  in  developing  training 
device  requirements  is  the  identification  of  a  variety  of 
training  options,  including  training  on  operational  equipment 
(Army  Training  Support  Center,  1988) .  However,  the  range  of 
devices  to  be  considered  is  often  restricted  by  premature 
commitment  to  a  specific  technology  or  device  option.  For 
example,  the  commandant  of  a  particular  school  might  attend  a 
device  demonstration  and  decide  that  the  school  needs  to  have  a 
device  like  the  one  demonstrated.  Engineers  provided  many 
examples  of  training  device  requirements  in  which  a  school 
specified  a  particular  device,  even  going  as  far  as  using  a 
corporate  brochure.  As  one  engineer  stated  "The  'Training 
Device  Requirement'  should  be  renamed  'Training  Requirement'. 

Too  often  the  TDR  lists  specifications  for  devices  but  not 
training  requirements."  The  artificial  restriction  of  device 
options  to  be  considered  influences  the  engineer's  job  in  one  or 
both  of  two  ways.  First,  the  engineer  may  spend  considerable 
time  and  effort  convincing  the  school  to  consider  other  device 
options.  Second,  if  the  engineer  is  unsuccessful  in  convincing 
the  school  to  expand  the  device  options,  the  TOD  and  TOA  process 
will  be  modified  by  PM  TRADE  and  TRADOC  schools  to  fit  a 
constrained  set  of  training  device  options. 
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Individual  Versus  Collective  Training 

We  found  that  the  application  of  training  devices  to 
collective  training  requires  engineers  to  address  unique 
variables.  For  example,  one  engineer  was  concerned  with  figuring 
out  how  a  device  could  be  used  to  train  soldiers  with  different 
Military  Occupational  Specialties  at  the  same  time.  The  engineer 
reasoned  that  the  equipment  status  required  to  support  the 
training  of  one  MOS,  such  as  allowing  a  soldier  to  turn  off 
electrical  power  to  troubleshoot  and  correct  a  simulated 
equipment  malfunction,  might  interfere  with  the  training  of  other 
MOSs.  In  the  case  of  the  power  example,  the  engineer  described  a 
situation  in  which  other  soldiers  would  be  unable  to  operate 
their  portion  of  the  equipment  while  the  power  was  off. 

In  addition  to  requiring  unique  considerations,  the  design  of 
collective  training  devices  involves  developing  methods  for 
designing  training  devices.  Although  the  Army  of  the  future  is 
expected  to  make  extensive  use  of  collective  training  devices  to 
complement  field  training  exercises  (Armstrong  and  Deaver,  1990) , 
the  Army  does  not  currently  have  a  substantial  set  of  guidelines 
for  developing  collective  training  devices.  The  application  of 
simulation  networking  (SIMNET)  to  collective  tactical  training, 
for  example,  is  only  a  few  years  old,  and  many  of  the  rules  for 
applying  SIMNET  to  collective  training  are  still  under 
development.  In  fact,  in  the  course  of  this  effort,  E  Division 
engineers  developed  and  implemented  a  data  collection  plan  as 
part  of  an  assessment  of  the  application  of  SIMNET  to  large  scale 
exercises.  The  lessons  learned  from  this  assessment  feed  into 
the  device  design  process  for  collective  training  devices. 

Extent  to  Which  a  Project  Involves  Novel  Applications 
of  Training  Technology 

When  a  project  involves  the  novel  application  of  training 
technology,  the  workload  of  an  engineer  increases  substantially, 
because  the  engineer  must  figure  out  the  methods  for  applying  the 
new  technology.  One  example  of  the  novel  application  of  training 
technology  is  the  application  of  embedded  training  (use  of 
operational  hardware,  training  software,  and  sometimes  auxiliary 
hardware  to  provide  training  in  comparison  with  stand  alone 
discrete  devices)  to  collective  tactical  training  requirements. 
The  fact  that  considerable  time  will  be  required  before  engineers 
know  what  information  about  training  requirements  they  will  need 
to  develop  a  materiel  solution  to  this  problem  can  be  appreciated 
when  one  considers; 

•  A  ten  volume  series  has  been  developed  on  the  application 
of  embedded  training  (Finley,  Alderman,  Peckham,  & 

Strasel,  1988),  but  does  not  address  the  issue  of 
collective  tactical  training. 
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•  The  Army  is  still  in  the  process  of  learning  how  to  apply 
training  devices  to  collective  tactical  training  through 
its  work  with  SIMNET. 

•  The  embedded  training  concept  has  been  divided  into  two 
classes;  devices  that  are  a  permanent  fixture  on  the 
operational  equipment,  and  devices  that  are  temporarily 
appended  to  operational  equipment  to  support  training 
exercises. 

Training  Device  Policy  Statements 

Training  policy  statements  influence  the  range  of  training 
device  options  to  be  addressed  in  TODs,  TOAs,  and  CTEAs.  They 
also  influence  the  weights  assigned  to  certain  variables  in 
conducting  TOAs  and  CTEAs.  The  effect  of  policy  statements 
on  CFP  has  recently  been  reported  (Hinton  et  al.,  1990).  For 
example,  a  policy  statement  signed  by  the  Vice  Chief  of  Staff  of 
the  Army  and  the  Under  Secretary  of  the  Army,  and  dated  3  March 
1987,  states  that  an  embedded  training  (ET)  "capability  will  be 
thoroughly  evaluated  and  considered  as  the  preferred  alternative 
among  other  approaches  to  the  incorporation  of  training 
subsystems  in  the  development  and  follow-on  Product  Improvement 
Programs  of  all  Army  materiel  systems"  (Department  of  the  Army, 
1987) .  Still  other  forms  of  policy  guidance  provide  goals  that 
are  specific  to  a  given  CFP  project.  Words  from  such  policy 
statements  influencing  current  PM  TRADE  projects  include:  "low- 
cost",  "modular",  and  "expandable." 

School  Versus  Unit  Training 

Consideration  of  the  environment  in  which  a  device  will  be 
employed  is  an  important  part  of  the  CFP.  The  environmental 
variables  mentioned  by  engineers  as  being  important  to  the  design 
of  training  devices  include  the  physical  resources  necessary  to 
support  the  equipment,  training  schedules,  skill  level  of  the 
soldiers  to  be  trained,  and  qualifications  of  trainers  and 
instructor/ operators.  Devices  destined  for  use  in  a  school 
are  designed  to  meet  a  carefully  described  training  environment, 
while  devices  destined  for  use  in  units  must  be  designed  to 
accommodate  a  wide  range  of  training  environments. 

The  difference  between  designing  a  device  for  a  school  versus 
unit  training  environment,  unlike  the  other  factors  influencing 
the  CFP,  was  not  identified  as  a  direct  result  of  data 
collection  within  PM  TRADE.  No  engineers  reported  problems  in 
obtaining  and  applying  information  about  the  unit  training 
environment;  however,  most  of  the  CFP  projects  involve  training 
devices  intended  to  be  used  in  an  institutional  setting. 
Information  obtained  outside  the  context  of  data  collection 
within  PM  TRADE  indicates  that  the  design  of  devices  for  use  in 
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an  institutional  setting  is  perceived  as  differing  from  the 
design  of  devices  for  use  in  a  unit  setting.  For  example,  the 
developers  of  OSBATS  state  that  this  tool  applies  to  designing 
devices  for  an  institutional  setting  (Sticha,  1990) ,  and  the 
developers  of  the  MANPRINT  Product  Four  tool  for  early  estimation 
of  training  requirements  for  new  weapon  systems  state  that  this 
tool  applies  to  institutional  training  (Ditzian,  Roth,  and 
Johnston,  1987) . 

Factors  That  do  not  Influence  the  Training  Device  CFP 

During  the  course  of  data  collection  it  became  apparent  that 
certain  factors  that  we  expected  to  have  a  substantial  impact  on 
the  jobs  of  engineers,  based  on  our  review  of  formal  regulations 
and  guidance  documents,  have  little  impact.  For  example,  the 
U.S.  Army  Training  Support  Center  (ATSC)  Training  Devices 
Management  Directorate  decided  that  differences  between  system 
and  nonsystem  device  design  warranted  the  publication  of  two 
separate  procedural  guides  for  training  device  requirements 
documentation,  one  for  system  devices  and  one  for  nonsystem 
(ATSC,  July,  1989;  September,  1989).  However,  engineers  report 
that  their  job  differs  little  between  these  two  types  of  devices. 
For  example,  there  are  a  variety  of  documents  that  schools  can 
use  to  describe  their  need  for  a  training  device  depending  upon  a 
number  of  factors.  Engineers  simply  refer  to  all  of  these 
documents  as  "requirements"  or  "the  user's  requirements."  In 
attempting  to  discuss  fine  points  of  the  various  types  of 
documents  with  engineers  it  became  apparent  that  even  some  senior 
engineers  were  confused  by  the  proliferation  of  document  titles. 

In  reviewing  CFP  regulations  and  other  guidance  one  notes  the 
frequent  mention  of  the  Life  Cycle  Systems  Management  Model 
(LCSMM)  with  its  three  decision  points  and  the  Army  Streamlined 
Acquisition  Process  (ASAP)  with  its  two  decision  points.  The 
distinction  between  LCSMM  and  ASAP,  like  that  between  system  and 
nonsystem  devices,  has  little  impact  on  the  job  of  engineers 
except  that  the  job  may  need  to  be  performed  more  quickly  under 
ASAP.  Names  of  requirement  documents  and  specifics  of  the  review 
process  might  change  as  a  function  of  differences  among  projects, 
but  the  job  of  the  engineer  remains  the  same;  make  sure  a  device 
is  truly  required,  and  look  for  a  materiel  solution  that  is 
effective,  affordable,  technically  feasible,  and  supportable. 

Issues  Raised  by  Engineers  Regarding  CFP 

The  requirement  to  conduct  a  TOA  and  a  CTEA  separately  raises 
quality  control  issues.  Separate  analyses  suggest,  for  example, 
that  cost  and  training  effectiveness  are  not  considered  during 
the  TOA  and  materiel  issues  are  not  directly  relevant  to  a  CTEA. 
At  least  some  engineers  think  that  all  issues  involving  potential 
trade-offs  need  to  be  considered  at  one  time.  This  idea  is  not 
entirely  in  conflict  with  regulations  or  other  sources  of 
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guidance,  because  they  indicate  that  information  is  exchanged 
among  these  analyses. 

Questions  were  also  raised  regarding  the  relationship  between 
the  training  device  CFP  and  subsequent  configuration  management 
of  the  device  design.  In  certain  cases,  engineers  said  that 
significant  changes  are  made  in  a  device  when  it  reaches 
configuration  management  that  seem  to  contradict  the  results  of 
the  CFP.  Configuration  management  per  se  is  outside  the  scope  of 
the  current  project,  but  the  relationship  between  configuration 
management  and  the  TOA,  CTEA,  and  BTA  warrant  future 
investigation . 

Data  Sources  Used  in  the  Training  Device  CFP 

No  formal  databases  are  used  to  support  the  technical 
decisions  made  by  engineers.  Databases  are  available  that 
are  indirectly  relevant  to  CFP  or  might  cover  a  portion  of 
CFP  under  some  situations,  but  these  are  not  employed.  One 
example  of  such  a  database  is  the  Training  Cost  Data  Enhancement 
System  (T-CODES)  database  developed  by  the  Training  Performance 
Data  Center  to  assist  in  estimating  the  cost  of  training  devices. 
More  will  be  said  about  databases  in  the  chapter  on  engineers' 
ideas  for  improving  the  CFP. 

The  data  sources  most  frequently  referred  to  by  other 
engineers  are  TRADOC  schools,  engineering  colleagues,  support 
service  contractors,  and  vendors.  Some  engineers  also  referred 
to  using  ARI  reports,  ARI  personnel  and  Human  Engineering 
Laboratory  (HEL)  personnel. 

Conclusions 


•  The  differences  and  relationships  among  TOA,  CTEA,  BTA, 
and  device  configuration  management  are  not  clearly 
defined  in  regulations  or  other  guidance  documents. 
Further,  engineers  have  raised  questions  about  how  these 
components  of  training  device  CFP  and  design  relate  to  one 
another. 

•  At  a  high  level  the  baseline  CFP  is  a  relatively  uniform 
series  of  events  that  reflects  CFP  regulations.  At  a 
lower  level  there  are  substantial  differences  among  the 
projects  in  terms  of  procedures  used  as  a  function  of 
five  variables;  whether  a  school  specifies  that  it  wants 
a  specific  training  device;  the  extent  to  which  the 
project  involves  novel  applications  of  training 
technology;  whether  the  project  addresses  individual 
training  or  collective  training;  training  device  policy 
statements;  whether  the  project  addresses  school  or 

unit  training. 
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•  Formal  and  informal  research  on  how  to  apply  new  training 
technologies  and/or  expand  the  application  of  more  mature 
technologies  is  a  significant  portion  of  CFP.  To  some 
extent  engineers  continually  develop  specific  CFP 
procedures. 

•  The  system  and  nonsystem  categorization  of  training 
device  projects,  as  well  as  the  ASAP  and  LCSMM 
categorization  of  projects,  have  little  effect  on 
the  nature  of  the  job  performed  by  engineers. 

•  There  is  no  formal  system  of  data  sources  available  to 
support  training  device  CFP. 

Recommendations 

•  A  working  group  composed  of  school  and  PM  TRADE 
representatives  should  define  the  expected  outcomes  of 
a  TOA  and  CTEA  and  assess  whether  two  separate  analyses 
are  required. 

•  A  task  force  within  PM  TRADE  should  examine  the 
relationship  among  TOA,  BTA  and  configuration 
management. 

•  The  factors  that  influence  how  CFP  subtasks  are 
performed  across  projects  and  the  effects  of  these 
factors  on  performance  should  be  defined  in  adequate 
detail  to  guide  efforts  to  assian  responsibilities  within 
training  device  CFP.  A  starting  point  for  this  effort 
might  simply  involve  distributing  copies  of  Appendix  C 

to  all  engineers  and  asking  them  to  add  or  delete 
subtasks  based  on  their  experience  with  the  CFP. 


Need  for  Training  Decision  Aids  Within  E  Division 


A  major  question  that  confronted  the  researchers  at  the 
beginning  of  this  project  was  whether  or  not  engineers  consider 
variables  that  influence  the  acquisition  and  retention  of  skills 
when  formulating  the  concept  for  a  training  device?  The  previous 
chapter  alluded  to  the  fact  that  engineers  do  indeed  consider 
these  variables.  This  chapter  attempts  to  express  the  depth  and 
scope  of  decisions  about  learning  variables  made  by  engineers, 
but  first  it  is  useful  to  discuss  an  important  lesson  learned 
about  the  perceived  difference  between  the  terms  ••training"  and 
••learning.  •• 

The  questionnaire  and  interview  process  made  extensive  use  of 
the  term  ••training"  to  refer  to  the  psychological  process  of 
acquiring  skills.  This  proved  to  be  a  poor  selection  of  terms, 
because,  by  mission  definition,  every  decision  made  within  PM 
TRADE  involves  consideration  of  training  variables.  Therefore, 
from  the  perspective  of  engineers,  the  term  "training^^  includes  a 
host  of  factors  that  are  distantly  related  to  skill  acquisition. 

Table  2  summarizes  the  results  of  a  questionnaire  item  in 
which  engineers  were  asked  whether  they  made  specified  decisions 
and  if  they  encountered  problems  in  making  each  decision. 
Responses  to  this  item  were  discussed  with  engineers  during 
interviews  to  gain  more  detailed  information  about  the  extent  to 
which  they  consider  learning  principles  in  their  work. 

Identifying  cue  and  response  fidelity  required  to  provide 
effective  training  appeared  to  be  the  most  important  training 
decisions  from  the  perspective  of  engineers,  because  fidelity 
issues  tended  to  be  raised  in  the  context  of  discussions 
regarding  many  of  the  decisions  listed  in  Table  2.  As  shown  in 
the  table,  more  engineers  reported  encountering  problems  in 
making  decisions  regarding  fidelity  requirements  than  reported 
making  these  decisions,  contrary  to  the  trend  shown  for  all  other 
types  of  decisions. 

"The  customer  always  asks  for  the  moon."  Disagreements 
between  training  device  designers  and  proponents  about  fidelity 
requirements  were  frequently  referred  to  by  the  engineers.  Many 
of  the  engineers  recognize  that  high  fidelity  is  not  always 
needed  in  training  simulations.  The  users  consistently  insist  on 
high  fidelity.  For  example,  command  and  control  training 
normally  does  not  require  a  high  level  of  fidelity.  However,  if 
the  user  wanted  to  make  the  training  model  comparable  to  combat 
development  models,  that  probably  should  be  high  fidelity.  •'High 
fidelity  requirements  are  often  overkill,  [the  proponent]  has 
lost  sight  of  the  training  requirement." 

Examples  were  also  given  of  arguments  about  fidelj't’y 
requirements  for  flight  simulators,  driving  simulators,  and  tank 
gunnery  simulators.  For  the  flight  simulator  the  proponent 


19 


TABLE  2.  NUMBER  OF  ENGINEERS  REPORTING  THAT  THEY  ARE  REQUIRED  TO 
TO  MAKE  VARIOUS  KINDS  OF  TRAINING  DECISIONS  AND  NUMBER  REPORTING 
PROBLEMS  IN  MAKING  THESE  DECISIONS 


Number  of  Number  of 
Engineers  Engineers 


Decision 

Making 

Decision 

Encountering 
Problems  in 
Making  Decision 

If  a  proposed  training  device 

would  duplicate  an  existing 

device  (or  device  under  development)? 

12 

10 

If  fidelity  requirements  (e.g. 
degrees  of  motion  for  a  flight 
simulator)  are  clearly  specified? 

12 

12 

If  fidelity  requirements  are  valid? 

12 

15 

How  much  it  will  cost  to  include 
a  particular  instructional  feature 
in  a  training  device? 

15 

12 

Which  instructional  support  features 
should  be  included  in  a  simulator 
or  embedded  training  application? 

11 

11 

Which  training  device  options  (e.g., 
full  mission  simulator, embedded 
training)  or  combination  of  options 
are  the  most/least  cost-effective? 

10 

9 

Whether  a  particular  training  device 
is  compatible  with  the  institutional 
or  unit  training  environment 

10 

10 

If  a  Trade  Off  Determination  (TOD) 

13 

12 

adequately  defines  the  issues  to 
be  addressed  in  a  Trade  Off 
Analysis  (TOA) 
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argued  for  graphics  with  a  level  of  visual  resolution  that  was 
much  higher,  and  more  expensive,  than  the  engineer  thought  was 
necessary.  For  the  driver  trainer  the  proponent  wanted  a  motion 
platform  with  six  degrees  of  freedom,  the  engineer  argued  that  a 
seat  shaker  would  be  adequate. 

In  each  of  these  examples  the  engineers  are  arguing  for  lower 
fidelity  and  the  resulting  significant  savings.  The  engineers 
point  out  that  they  do  not  have  ready  access  to  published 
research  or  accepted  training  effectiveness  models  to  support  the 
argument  for  using  lower  fidelity  in  training  simulators. 

Engineers  were  asked  to  list  what  they  consider  to  be  the 
most  important  training  decisions  influencing  the  selection  of  a 
Best  Technical  Approach  (BTA) .  The  responses  to  this  question 
were  quite  diverse.  Four  engineers  reported  that  deciding 
fidelity  requirements  was  among  the  most  important  training 
decisions,  and  two  engineers  reported  that  deciding  how  many 
training  devices  are  required  is  one  of  the  most  significant 
training  decisions.  These  were  the  only  clear-cut  instances 
where  there  were  any  agreements  upon  significant  training 
decisions.  Further,  certain  of  the  responses  (How  much  will  the 
device  cost?  How  much  does  the  school  want  to  spend  and  when?) 
were  unrelated  to  the  learning  process.  However,  the  reader 
should  keep  in  mind  the  problems  with  using  the  word  "training" 
within  PM  TRADE  mentioned  at  the  beginning  of  this  chapter. 

Engineers  were  also  asked  "what  types  of  training  or  learning 
issues  do  you  consider  in  your  work?"  The  responses  to  this 
question  were  as  diverse  as  those  to  the  question  about  the  most 
significant  training  decisions  that  bear  upon  the  BTA.  Below  is 
an  edited  list  of  training  considerations,  based  upon  responses 
to  two  questions  about  types  of  learning  issues  and  important 
training  decisions  in  the  BTA. 

•  Is  the  device  needed? 

•  Who  will  be  trained? 

•  What  tasks  will  be  trained? 

•  What  types  of  skills  are  involved  in  performing  the  tasks 
(e.g.,  motor  versus  cognitive)? 

•  Where  will  the  training  be  conducted? 

•  When  will  the  training  be  conducted? 

•  What  is  the  goal  of  the  training? 

•  How  easy  to  maintain  is  the  device? 
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•  What  is  the  minimum  degree  of  fidelity  that  will 
support  effective  training? 

•  How  does  the  soldier  interact  with  the  equipment? 

•  How  will  performance  be  scored? 

•  How  will  After  Action  Reviews  (AARs)  be  conducted? 

•  Does  the  device  reflect  a  knowledge  of  training 
principles  (reinforcement,  progressive  training, 
corrective  feedback,  etc.)? 

•  What  training  methods  should  be  employed  (e.g.,  train 
entirely  by  a  single  trainer,  part  task  trainers  or  a 
combination  of  the  two?)? 

•  Is  the  device  part  of  an  integrated  training  strategy? 

•  How  well  does  a  particular  alternative  achieve  the  desired 
training  objectives  versus  life  cycle  cost? 

•  Who  will  support  this  device  in  the  field? 

•  How  many  training  devices  are  required? 

•  Is  the  training  device  needed? 

•  How  will  the  device  be  used  or  interfaced  with  other 
equipment? 

•  What  existing  resources  are  available. .. "as  is"  or  easily 
modified  to  meet  the  training  requirements? 

•  How  effective  are  training  device  options? 

•  What  are  the  performance  standards? 

•  Does  the  device  duplicate  an  existing  device? 

In  the  course  of  interviewing  individual  engineers  it  became 
apparent  that  there  are  large  differences  in  terms'  of  the  extent 
to  which  they  considered  training  and  learning  variables  during 
device  design.  At  one  end  of  the  training  consideration 
continuum  are  those  engineers  who  believed  that  any  device  that 
meets  fidelity  requirements  is  an  effective  trainer.  Further 
along  the  continuum  are  engineers  who  are  concerned  with  issues 
like  how  to  provide  feedback  to  students.  Yet  further  along  the 
continuum  are  those  engineers  who  grapple  with  such  problems  as 
integrating  individual  skills  training  with  collective  training 
in  a  single  device.  This  sophisticated  end  of  the  continuum  is 
illustrated  by  the  comments  provided  below. 
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Which  instructional  support  features  should  be  included  in  a 
simulator  or  embedded  training  application?  ”[We]  always  try  to 
reduce  the  instructor  load.  Instructors  do  not  realize  how  much 
they  are  not  doing  until  you  free  them  from  some  responsibility 
by  using  CAI  [computer  assisted  instruction],  CMI  [computer 
managed  instruction],  etc.,  these  features  allow  them  to  do 
other  things.”  The  same  engineer  who  made  this  statement  also 
pointed  out  that  instructional  support  features  help  to 
standardize  training  across  instructors. 

"The  Army  overall  and  we  as  an  organization  are  woefully 
delinquent  in  'productizing'  the  instructional  elements 
of  devices  and  so  I  am  beginning  to  think  in  terms  of  "teaching 
machines"  opposed  to  training  devices."  As  indicated  by  an 
additional  statement  made  by  this  engineer,  the  view  expressed  by 
this  statement  involves  examining  the  entire  learning  process  to 
decide  how  the  device  fits  into  the  learning  process.  "The 
system  engineer  needs  to  have  a  context  perspective  of  the 
training  objectives (s) ,  training  audience  and  intended  use 
(operational  environment)  of  the  device  or  simulator." 

Throughout  the  interviews,  and  during  project  meetings  with 
engineers,  many  other  examples  surfaced  where  engineers 
considered  variables  influencing  skill  acquisition  and  retention. 
One  engineer  was  concerned  with  how  to  integrate  the  training  on 
a  collective  training  device  of  soldiers  representing  multiple 
duty  positions  without  detracting  from  the  ability  to  meet  either 
individual  task  or  collective  task  training  objectives.  Another 
engineer  wrestled  with  the  problem  of  deciding  how  a  vehicle 
commander  might  be  both  a  trainer  and  a  trainee  if  a  crew 
training  device  were  embedded  in  a  vehicle.  Yet  another  engineer 
was  concerned  with  the  problem  of  taking  the  results  of 
performance  measures  and  organizing  the  results  of  the  measures 
in  a  way  that  could  be  used  to  provide  effective  feedback. 

No  set  of  procedures  is  used  uniformly  by  engineers  to  make 
decisions  that  require  consideration  of  learning  variables,  and 
no  such  set  of  procedures  exist.  As  mentioned  in  the 
introduction  to  this  chapter,  formal  regulations  on  training 
device  CFP  address  the  process  of  using  learning  relevant 
information  about  tasks  to  make  a  decision  about  training  device 
hardware  and  software  requirements  at  a  very  general  level.  Many 
of  the  decisions  described  above  are  not  covered  by  formal 
regulations,  but  formal  guidance  in  the  foim  of  research  products 
addresses  certain  portions  of  the  decision  process  in  detail. 

For  example,  the  second  volume  of  a  ten  volume  series  on  the 
application  of  embedded  training  addresses  the  process  of 
deciding  which  weapon  subsystems  and  tasks  are  good  candidates 
for  embedded  training  (Strasel,  Dyer,  Roth,  Alderman,  and  Finley, 
1988) ,  and  the  OSBATS  (Sticha,  1990)  addresses  the  process  of 
comparing  the  relative  cost-benefits  of  using  various  mixes  of 
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training  devices  (full  mission  simulators,  part  task  trainers, 
and  operational  equipment).  The  lack  of  an  overall,  integrated 
set  of  procedures  is  demonstrated  by  the  fact  that  there  are  no 
relations  linking  the  embedded  training  decisions  to  the 
decisions  facilitated  by  the  OSBATS  (Meliza  and  Knerr,  1990) . 

The  process  of  deciding  how  to  address  training  requirements 
with  a  materiel  solution  is  not  driven  by  available  information 
concerning  learning  variables.  Instead,  it  is  driven  by  an 
engineer's  recognition  of  the  fact  that  specific  learning 
variables  have  to  be  considered  when  examining  materiel  solutions 
to  a  training  requirement.  The  engineer  identifies  information 
needs  and  attempts  to  draw  this  information  from  schools  or  other 
organizations.  The  ATSC  Training  Devices  Management  Division  is 
attempting  to  modify  this  situation  somewhat  by  enhancing  the 
information  provided  by  schools  regarding  training  requirements, 
such  as  calling  for  training  strategies  (Army  Training  Support 
Center,  1988) . 

Further,  there  are  training  decisions  that  engineers  may  be 
in  a  better  position  to  make  than  are  TRADOC  schools.  These 
decisions  concern  the  application  of  specific  training 
technologies.  It  is  difficult  to  envision  a  situation  where  each 
branch  school  would  have  the  resources  necessary  to  track  these 
technologies  and  consider  their  potential  applications  to 
training.  PM  TRADE'S  ET  Branch  devote  their  efforts  to  this  area 
on  a  near  full  time  basis,  and  many  engineers  involved  directly 
in  concept  formulation  also  devote  some  of  their  time  to  this 
area. 

Conclusions 


•  Engineers  make  decisions  that  involve  consideration  of 
learning  issues. 

•  Engineers  vary  in  terms  of  their  appreciation  of  the 
importance  of  learning  variables  in  device  design. 

•  The  learning  variables  that  are  considered  most 
universally  by  engineers  are  cue  and  response  fidelity 
requirements,  and  engineers  want  more  information  about 
these  requirements  than  is  currently  available. 

•  No  systematic  process  is  used  to  insure  that 
learning  variables  are  considered  in  device  design. 

•  No  overall  systematic  process  even  exists  to  insure  that 
learning  variables  are  considered  in  device  design, 

•  Learning  variables  to  be  considered  vary  to  some  unknown 
degree  among  projects. 
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•  The  process  of  developing  information  about  training 
requirements  must  be  driven,  to  an  extent,  by  the 
information  needs  of  engineers. 

•  The  information  needs  of  engineers  regarding  training 
requirements  varies  among  projects  and  over  time  as 
new  training  technologies  become  available. 

Recommendations 


•  Learning  variables  must  be  linked  to  engineering 

variables  to  help  insure  that  adequate  information  about 
training  requirements  is  available  to  guide  training 
device  design.  This  recommendation  concerns  a  method  for 
linking  these  variables  and,  at  the  same  time,  it  would 
help  to  insure  a  uniform  and  high  level  of  appreciation 
of  the  importance  of  learning  variables  in  device  design. 
It  is  recommended  that  a  series  of  brief  (thirty  minutes 
or  less)  presentations  be  made  to  engineers  concerning 
major  topics  in  training  (such  as  the  integration  of 
individual  and  collective  training,  combined  arms 
training) .  Engineers  would  be  asked  to  respond  to  these 
presentations  by  pointing  out  if  and  how  the  information 
presented  would  be  used  to  guide  engineering  decisions. 
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Information  Flow 


A  problem  in  information  flow,  inadequate  description  of 
training  requirements,  has  been  noted  in  the  past  two  chapters  of 
this  report.  This  chapter  provides  additional  information 
relevant  to  this  problem,  describes  information  flow  between  PM 
TRADE  and  other  organizations  involved  in  training  device 
development,  and  discusses  information  flow  within  PM  TRADE. 

School  Descriptions  of  Requirements  for  Training  Devices 

Comments  made  by  engineers  helped  to  clarify  the  nature  and 
causes  of  problems  in  the  information  provided  by  schools  about 
training  requirements.  Two  types  of  problems  tend  to  surface, 
in  isolation  or  in  combination,  with  the  training  requirements 
prepared  by  schools.  First,  schools  frequently  attempt  to 
specify  training  devices  prematurely.  "The  'Training  Device 
Requirement'  should  be  renamed  'Training  Requirement'.  Too  often 
the  TDR  lists  specifications  for  devices  but  not  training 
requirements."  Second,  schools  do  not  consistently  describe 
training  requirements  in  a  manner  that  meets  the  information 
needs  of  engineers.  Further,  in  the  course  of  communicating  with 
schools  about  device  requirements,  engineers  frequently  discover 
that  no  task  analysis  or  media  analysis  has  been  performed  by  the 
school . 

The  ATSC  Training  Devices  Management  Division  is  aware  of  the 
problem  in  the  quantity  and  quality  of  information  provided  by 
schools  in  the  various  documents  they  prepare  to  describe  their 
device  needs.  ATSC  has  prepared  a  number  of  procedural  guides 
intended  to  aid  schools  in  preparing  these  documents  (U.S.  Army 
Training  Support  Center,  1988,  1989a,  1989b). 

The  inadequacy  of  the  information  provided  by  schools  is  due, 
in  some  cases,  to  the  fact  that  the  weapon  systems  on  which 
soldiers  are  to  be  trained  are  themselves  being  defined  through 
weapon  system  CFP.  More  than  one  engineer  mentioned  that  schools 
said  they  could  not  identify  and  describe  the  tasks  to  be  trained 
until  they  had  more  information  about  the  design  features  of  a 
new  weapon  system. 

Disseminating  Information  to  Industry  Regarding  Training 
Technology  Research  and  Development  Needs 

An  important  part  of  the  job  of  E  Division  is  the 
dissemination  of  information  to  industry  and  universities  about 
technological  needs,  and  E  Division  employs  a  variety  of 
mechanisms  for  influencing  future  training  technology.  E 
Division  engineers  present  papers  at  professional  meetings  such 
as  the  Interservice/Industry  Training  Systems  Conference 
describing  needs  for  training  technology  research.  Through 
membership  on  interservice  panels,  E  Division  helps  to  define 
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standards  for  training  technology  such  as  the  standards  for 
digitized  terrain  databases  and  simulation  networking  protocols. 

PM  TRADE  is  also  a  direct  sponsor  of  training  technology 
research.  For  example,  the  PM  TRADE/ ARI  Broad  Agency 
Announcement  (BAA)  describes  training  technology  issues  that 
warrant  being  addressed  on  a  high  priority  basis,  and  research 
and  development  organizations  submit  proposals  for  conducting 
research  addressing  these  issues.  Finally,  E  Division  helps  to 
guide  training  technology  research  through  formal  review  of 
research  proposals  and  progress  reports  for  projects  funded  by 
other  organizations.  One  example  of  this  process  is  the  review 
of  industry  sponsored  research  as  described  by  Independent 
Research  and  Development  (IRAD)  reports.  Another  example  is  the 
review  of  research  proposals  funded  by  the  Florida  Council  for 
High  Technology. 

This  project  did  not  include  an  assessment  of  the  quality  of 
guidance  provided  by  PM  TRADE  regarding  research  and  development 
needs.  However,  the  extent  to  which  this  guidance  helps  to  focus 
limited  resources  on  addressing  critical  needs,  as  opposed  to 
"nice  to  have"  products,  is  important. 

Integration  of  ET  Branch  Functions  With  the  Day  to  Day  CFP 

Problems  with  the  flow  of  information  within  E  Division  were 
noted  in  terms  of  the  perceived  relevance  of  ET  Division  to  day 
to  day  CFP  operations.  A  substantial  portion  of  the  work  of  ET 
Division  is  concerned  with  guiding  the  development  of  technology 
to  meet  future  training  device  needs.  These  individuals  tend  to 
work  in  one  or  more  specific  technical  areas  involving 
technologies  that  involve  developmental  risks.  This  work  often 
does  not  have  an  immediate  payoff  from  the  perspective  of 
engineers  working  on  concept  formulation  projects  who  want  to 
know  what  technologies  are  available  for  application  in  the 
present. 

The  expertise  ET  personnel  have  gained  in  various  areas  of 
training  technology  would  allow  them  to  make  important 
contributions  to  CFP  projects,  and  some  engineers  do  seek 
information  from  ET  personnel  for  application  to  CFP  projects. 
However,  as  a  general  rule,  ET  personnel  are  not  tasked  to  work 
on  CFP  project  teams,  thereby  reducing  potential  interactions 
between  ET  personnel  and  engineers  working  on  CFP  projects. 

For  example,  one  instance  was  observed  where  two  individuals  from 
ET  had  gained  extensive  research  experience  with  a  particular 
training  device.  When  a  training  device  requirement  was  received 
which  described  the  possibility  of  using  a  modified  form  of  the 
device  to  meet  the  needs  of  a  school,  these  personnel  from  ET 
branch  were  not  tasked  to  take  part  in  the  initial  review  of  the 
requirement. 
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Feedback  Regarding  Training  Device  Effectiveness 

There  is  no  formal  mechanism  for  providing  E  Division 
engineers  with  feedback  regarding  the  effectiveness  and 
utilization  of  training  devices.  E  Division's  role  in  the 
development  of  a  particular  device  is  generally  completed  long 
before  the  device  undergoes  operational  testing,  and  the 
operational  testing  and  evaluation  that  is  conducted  is  concerned 
with  assessing  whether  the  device  meets  technical  specifications. 
Assessing  how  well  a  device  supports  skill  acquisition  and 
retention  is  not  consistently  addressed  by  operational  testing. 

Information  about  training  device  utilization  is  provided  for 
use  by  PM  TRADE  in  a  hit  or  miss  fashion.  For  example,  many 
engineers  within  PM  TRADE  know  that  instructional  support 
features  for  simulators  are  often  underutilized  in  institutions. 
However,  there  are  little  data  that  can  be  used  to  prove  that 
specific  instructional  features  are  likely  to  be  underutilized. 

Conclusions 


•  The  most  severe  problem  with  information  flow  concerns 
information  provided  by  schools  regarding  training 
requirements.  Engineers  want  more  information  from 
schools  a\  ^  ,  device  requirements.  In  particular, 
engineer's  want  schools  to  specify  the  tasks  to  be  trained 
and  provj.de  information  about  cue  and  response  fidelity 
requ: rements. 

•  Our  description  of  the  information  flow  between  E 
Division  and  schools  is  only  partial,  in  that  it  is  not 
based  upon  input  from  school.  Until  school  input  is 
obtained  regarding  this  problem,  it  is  premature  to 
propose  a  solution. 

•  In  general,  members  of  ET  Branch  are  not  assigned  to  work 
on  concept  formulation  projects.  This  reduces  potential 
interactions  between  ET  Branch  and  engineers  engaged  in 
the  CFP  on  a  daily  basis. 

•  E  Division  employs  a  variety  of  mechanisms  to  help  guide 
training  technology  research  and  development. 
Participation  of  ET  personnel  on  CFP  projects  would  help 
to  insure  that  the  work  they  conduct  in  training 
technology  research  fits  the  information  and  product 
needs  of  engineers  working  on  device  design  efforts.  On 
the  other  hand,  tasking  ET  personnel  to  work  on  CFP 
projects  runs  the  risk  of  creating  a  situation  where 
these  personnel  would  end  up  performing  tasks  that  would 
otherwise  be  performed  by  other  engineers  at  the  expense 
of  ET's  unique  mission. 
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Recommendations 


•  The  policy  regarding  assignment  of  ET  personnel  to  CFP 
should  be  refined  to  allow  these  engineers  to  contribute 
their  knowledge  of  specific  areas  of  training  technology. 
However,  this  involvement  in  projects  should  be  carefully 
controlled  to  prevent  distraction  of  these  personnel  from 
ET  branch's  unique  mission. 

•  ET  Branch  should  make  aggressive  attempts  to  keep 
engineers  from  ED  and  EC  Branch  informed  of  the  projects 
on  which  they  are  working.  At  the  same  time,  ED  and  EC 
Branch  should  make  aggressive  attempts  to  keep  ET  Branch 
aware  of  training  technology  issues  that  surface  during 
the  training  device  CFP. 

•  Conduct  of  the  training  device  CFP  should  be  examined 
from  the  perspective  of  Army  schools  and  from  the 
perspective  of  other  PM  TRADE  divisions. 
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Engineers'  Ideas  for  Improving  CFP 


Responses  to  the  six  questions  in  the  "Career  Development  and 
Job  Satisfaction"  portion  of  the  questionnaire  were  used  to  gain 
information  about  interventions  that  might  help  support  CFP. 
During  interviews,  engineers  were  asked  if  they  had  any 
suggestions  for  improving  the  training  device  CFP.  Discussions 
of  the  preliminary  results  of  this  project  tended  to  elicit 
disagreements  about  certain  obstacles  to  CFP  and  the  proposals 
for  addressing  these  obstacles.  These  disagreements  helped  to 
clarify  the  nature  of  the  obstacle  and  provide  guidance  regarding 
possible  interventions. 

Inadequate  description  of  training  requirements  was  viewed  as 
a  major  obstacle  within  CFP.  A  few  ideas  for  addressing  this 
obstacle  involved  the  employment  of  the  U.S.  Army  Research 
Institute  (ARI) .  One  proposal  concerned  ARI  developing  a 
decision  support  system  that  would  help  schools  develop  training 
requirements.  A  second  proposal  involved  ARI  working  directly 
with  schools  in  preparing  training  requirements. 

Other  ideas  for  improving  the  CFP  involved  developing 
databases  to  support  CFP  and  refining  CFP  procedures.  These 
ideas  are  presented  and  explained  below.  Statements  in  quotation 
marks  were  made  by  engineers. 

Develop  Databases  to  Support  CFP 

Formal  databases  are  not  used  to  support  technical 
decisions  made  during  CFP.  The  databases  engineers  believe  might 
aid  them  in  the  training  device  CFP  include  training  technology 
databases,  cost  databases,  historical  databases,  and  procedural 
databases.  In  some  cases  engineers  specified  automated 
databases,  and  in  other  cases  they  merely  defined  the  need  for  an 
organized  and  accessible  body  of  information. 

"Most  places  that  do  engineering  have  a  library  of  tech 
books,  catalogs  or  something.  We  have  nothing,  absolutely  zero. 
The  old  TRADE  location  had  old  references  that  were  better  than 
nothing.  We  need  an  in-house  database  and  tech  library." 

There  are  databases  to  which  engineers  have  access,  but  they 
are  not  used  to  support  CFP.  The  reasons  why  existing  databases 
are  not  used  are  important,  because  they  help  to  define  the 
characteristics  of  a  database  that  engineers  will  choose  to  use. 
The  only  database  that  contains  training  technology  information 
is  the  Independent  Research  and  Development  (IRAD)  database 
maintained  by  ET  Branch.  This  database  covers  only  those 
projects  being  conducted  in-house  by  industry,  and  it  is  rarely 
used  by  engineers  during  CFP.  The  IRAD  database  is  not  used  by 
engineers  during  concept  formulation,  because  they  are  expected 
to  propose  technologies  that  have  low  or  no  developmental  risks, 
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and  the  risks  associated  with  many  of  the  IRAD  projects  are 
perceived  as  being  high. 

Engineers  reported  that  cost  databases  would  be  useful, 
because  it  is  difficult  to  obtain  cost  data.  The  costs  of 
instructional  features  change  rapidly,  and  it  is  especially 
difficult  to  estimate  the  costs  of  features  that  differ  from 
those  used  on  previous  trainers.  Engineers  have  access  to  a  cost 
database  known  as  T-CODES  that  is  maintained  by  the  Training 
Performance  Date  Center,  but  only  one  engineer  mentioned  using 
this  aid.  That  engineer  concluded  that  T-CODES  is  a  start  in  the 
right  direction  but  "it  is  at  too  high  of  a  level."  Cost 
estimates  need  to  be  broken  down  into  functional  areas  (e.g., 
costs  of  different  kinds  of  visual  systems)  to  show  costs  of 
single  trainers  and  single  instructional  features. 

Engineering  colleagues  are  one  of  the  most  important  sources 
of  information  used  by  engineers.  A  problem  that  faces  engineers 
when  attempting  to  identify  colleagues  with  experience  in  a 
particular  area  is  the  lack  of  a  central  and  easily  accessible 
source  of  information  about  who  has  experience  in  specific 
subject  areas.  ET  Branch  recognizes  the  importance  of  colleagues 
as  data  sources  and  is  developing  an  automated  data  base  to  help 
engineers  identify  colleagues  with  experience  in  particular 
areas.  Also,  ET  branch  is  beginning  to  address  the  need  for  a 
training  technology  database.  Under  the  guidance  of  ET  Branch, 
the  Institute  for  Simulation  and  Training  (1ST)  is  defining 
functional  requirements  for  such  a  database. 

Refine  CFP  Procedures 


Engineers  presented  diverse  ideas  regarding  the  need  to 
refine  CFP  procedures.  Some  refinements  simply  involve 
separating  guidance  for  the  training  device  CFP  from  guidance  for 
the  prime  weapon  system  CFP,  since  most  regulations  and  other 
sources  of  guidance  attempt  to  address  both  types  of  efforts  with 
identical  guidance.  Other  refinements  involve  developing 
procedures  based  on  the  application  of  systems  engineering, 
refining  procedures  to  make  the  CFP  be  more  efficient,  and 
developing  procedures  for  applying  specific  training 
technologies.  Each  of  these  types  of  ideas  for  refining 
procedures  is  discussed  below. 

More  than  one  engineer  pointed  out  the  fact  that  most 
regulations  addressing  concept  formulation  are  based  on  the  CFP 
for  prime  weapon  systems  rather  than  training  devices. 

For  this  reason,  among  others,  there  is  a  lack  of  regulation- 
based  procedures  for  developing  concepts  for  training  devices. 
Some  engineers  also  believe  that  many  of  the  paperwork 
requirements  that  make  sense  when  executing  the  CFP  for  a  prime 
weapon  system  are  inappropriate  for  training  device  CFP. 
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"There  is  a  need  for  a  systems  engineering  methodology. 

Choose  a  method,  establish  it,  train  it,  use  it  on  a  day  to  day 
basis.  Otherwise,  individuals  will  vary,  will  not  produce 
consistent  results  or  v;ill  not  produce  results  at  all." 

Engineers  also  expressed  a  need  for  refining  procec  ares  to 
enhance  efficiency.  "The  acquisition  process  does  not  tolerate 
time  pressure.  The  process  must  be  fixed,  and  new  ways  are 
needed  to  do  things  faster."  There  is  a  "lack  of  enough  time  or 
personnel  to  do  a  job  right." 

Engineers  also  presented  arguments  against  further  specifying 
CFP  procedures.  Some  engineers  expressed  the  opinion  that 
project  diversity  would  make  it  impossible  to  develop  specific 
procedures  that  appZy  to  all  projects.  The  net  result 
of  further  specificai  ion  of  procedures  might  be  a  proliferation 
of  unnecessary  paperwork  requirements.  However,  careful 
consideration  of  the  important  differences  among  projects,  such 
as  those  described  in  the  chapter  on  baseline  CFP,  should  help  to 
tailor  the  application  of  refined  procedures  to  those  projects 
for  which  they  are  indeed  appropriate. 

Another  argument  against  further  specification  of  CFP 
procedures  is  in  terms  of  damage  to  career  development,  skills 
and  expertise.  Certain  PM  TRADE  engineers  are  of  the  opinion 
that  any  engineer  who  needs  detailed  procedures  is  not  qualified 
to  perform  his  or  her  job.  This  outlook  is  by  no  means  unique  to 
PM  TRADE  or  to  the  engineering  profession.  Certain  supervisors 
and  job  performers  consider  the  use  of  job  aids  as  an  indication 
that  the  user  lacks  critical  skills  (Chenzoff,  Joyce  and  Nauta, 
1985) ,  and  Riedel  (1988)  has  noted  that  this  perceived  threat  to 
skills,  expertise,  career  or  status  is  a  threat  to  the  acceptance 
of  job  aids. 

Fortunately,  the  situation  within  PM  TRADE  is  of  a  type  that 
bodes  well  for  the  acceptance  of  aids.  Post  and  Price  (1973) 
listed  three  characteristics  of  a  good  work  environment  that 
appear  to  be  particularly  relevant  to  the  acceptance  of  decision 
support  systems  or  other  aids.  A  good  aid  is  one  that  affords 
the  opportunity  to  learn  skills  or  knowledge  relevant  to  career 
advancement,  challenges  individuals  in  terms  of  quantity  or 
complexity  of  job  tasks,  and  gives  individuals  a  significant  role 
in  performing  tasks  perceived  to  be  critical  to  the  mission  of 
their  organization.  The  size  and  complexity  of  many  of  the 
PM  projects,  combined  with  the  need  to  make  rapid  decisions, 
result  in  CFP  ending  when  time  runs  out,  rather  than  ending  when 
engineers  believe  all  issues  have  been  addressed.  Procedural 
refinements  offer  the  potential  of  helping  engineers  address  more 
of  their  tasks  in  a  given  amount  of  time. 

One  engineer  made  an  important  point  about  the  relation 
between  refining  the  CFP  and  applying  automation  to  CFP. 
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Automating  the  current  CFP  would  be  inappropriate  and  result  in  a 
decision  support  system  of  limited  utility.  "If  we  are  going  to 
do  concept  formulation  in-house,  we  will  need  tools.  But  first 
we  must  refine  the  process." 

Project  Integration 

There  is  a  "Need  for  a  planning  process  which  provides  an 
opportunity  to  integrate  individual  activities  and  projects  into 
investment  strategies  which  match  Army  training  strategies." 

Many  of  the  projects  addressed  by  E  Division  engineers  are 
expected  to  overlap  in  terms  of  functional  requirements, 
suggesting  the  possibility  of  integrating  these  projects  to 
increase  their  cost-effectiveness.  Other  projects  requiring 
integration  are  those  resulting  in  training  devices  to  be  used  in 
an  integrated  fashion.  The  current  workload,  combined  with  other 
factors  (e.g.,  differences  among  projects  in  terms  of  schedules, 
funding,  and  proponents)  makes  it  difficult  to  take  advantage  of 
these  possibilities. 

In  the  course  of  participating  in  various  project  meetings 
within  PM  TRADE,  we  observed  on-the-spot  attempts  to 
integrate  projects  involving  similar  functional  requirements 
and/or  training  devices  whose  use  need  to  be  integrated  in 
schools  or  units.  In  one  case,  PM  engineers  noted  the  similarity 
in  requirements  between  two  projects  involving  instrumented 
training  for  Battalion  Task  Force  level  field  exercises. 

Engineers  expended  considerable  effort  attempting  to  define  the 
benefits  of  integrating  these  projects  to  proponents  concerned 
with  differences  between  projects  in  terms  of  funding  sources  and 
milestones.  In  another  case,  engineers  raised  questions  about 
how  the  application  of  simulation  networking  to  combined  arms 
training  in  the  form  of  the  Close  Combat  Tactical  Trainer  (CCTT) 
is  to  be  integrated  with  combined  arms  training  at  the 
National  Training  Center  (NTC) .  Is  there  an  Army  strategy  for 
linking  CCTT  training  and  training  at  NTC  that  needs  to  be 
considered  in  the  design  of  CCTT? 

Training 

Engineers  were  also  asked  what  training  they  would  like  to 
have  to  help  them  do  a  better  job.  Responses  to  this  question 
included  system  engineering  training  (general  or  specific  to 
training  device  development) ,  cost  estimation  training,  training 
and  workshops  on  the  application  of  new  technology,  instruction 
on  learning  theory,  computer  training,  software  training,  a 
course  on  Army  maneuver  tasks,  and  contracting  courses. 

Conclusions 


•  Cost,  historical,  training  technology  and  procedural 
databases  would  be  useful,  but  they  must  be  tailored  to 
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meet  the  information  needs  of  engineers  in  a  direct 
manner.  Any  attempt  to  develop  databases  should  include 
sustained  participation  by  engineers  (rather  than  a  one 
shot  needs  analysis)  to  insure  that  the  final  product 
directly  meets  their  information  needs. 

•  Any  attempt  to  automate  the  CFP  procedures  should  wait 
until  procedural  refinements  are  accomplished,  or  the 
automation  effort  should  be  conducted  in  tandem  with  the 
effort  to  refine  procedures. 

•  Any  attempt  to  refine  CFP  procedures  must  consider 
differences  among  projects  in  order  to  support  individual 
engineers  more  effectively  and  avoid  creating 
unnecessary  paperwork. 

Recommendations 


•  Engineers  should  be  assigned  to  work  with  any  group 
attempting  to  design  databases  for  use  by  E  Division. 

•  Once  again,  the  five  factors  identified  in  the  chapter 
on  baseline  CFP  as  influencing  CFP  procedures  should  be 
examined  in  detail  to  find  out  how  they  influence 
information  needs  and  job  tasks. 

•  PM  TRADE  should  consider  preparing  a  white  paper  on  how 
projects  might  be  integrated,  the  benefits  to  be  gained 
from  project  integration,  and  impediments  to  project 
integration. 

•  The  U.S.  Army  should  consider  funding  policies  that 
provide  schools  and  PM  TRADE  with  greater  flexibility 
to  integrate  related  projects. 
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Fit  of  the  OSBATS  With  Training  Device  CFP 

ARI  and  PM  TRADE  established  a  research  program  to  develop 
methods  for  designing  cost-effective  training  devices.  As  part 
of  this  program,  a  decision  aid  was  developed  to  assist  the 
training  device  designer  in  performing  TOAs  for  the  training 
device  CFP.  This  decision  aid  is  called  the  OSBATS.  It  is  a 
computer-based  model  that  interactively  uses  rule-based 
procedures  and  databases  to  support  that  portion  of  the  concept 
development  of  training  devices  concerned  with  the  TOA  (Singer 
and  Sticha,  1987) .  The  goals  of  OSBATS  are  to  specify  the  design 
and  use  of  training  equipment  to  satisfy  training  requirements  at 
a  minimum  cost,  or  to  provide  the  greatest  training  effectiveness 
for  a  given  cost  (Sticha,  Blacksten,  et  al.,  1990). 

OSBATS  was  developed  using  a  top-down  approach.  That  is,  an 
overall  goal  was  specified,  the  goal  was  then  decomposed  into 
subgoals,  and  procedures  were  developed  for  achieving  the 
subgoals.  The  developers  were  not  constrained  to  make  OSBATS 's 
procedures  compatible  with  existing  device-design  practice.  The 
use  of  the  top-down  approach,  in  fact,  nearly  guaranteed  that  new 
device-design  procedures  would  result.  The  OSBATS  developers 
also  were  not  constrained  to  limit  data  requirements  for  the  new 
model  to  the  data  requirements  of  the  current  design  process. 

This  chapter  is  concerned  with  comparing  the  functions  served 
by  OSBATS  with  the  training  device  CFP  to  provide  input  for  an 
ongoing  project  to  refine  the  OSBATS.  This  comparison  is 
necessary,  in  part,  because  OSBATS  guidance  does  not  explain  how 
it  is  intended  to  fit  into  the  CFP,  other  than  indicating  that  it 
was  designed  to  support  the  TOA. 

Fit  of  the  OSBATS  With  Major  CFP  Tasks 

The  major  CFP  tasks  are  establishing  a  dialogue  between 
training  developers  and  materiel  developers,  performing  a  TOD, 
performing  a  TOA,  performing  a  CTEA,  developing  a  BCE,  and 
developing  a  BTA.  A  potential  problem  is  that  the  OSBATS  was 
designed  to  support  only  one  of  these  tasks,  and  yet  all  of  these 
tasks  are  inter-related. 

Since  the  scope  of  the  OSBATS  is  the  conduct  of  a  TOA,  the 
model  assumes  implicitly  that  all  of  the  information  to  be 
derived  from  the  specification  of  training  requirements  and  the 
performance  of  a  TOD  are  available  for  incorporation  within  the 
OSBATS  database.  The  OSBATS  model  also  assumes  that  all  of  the 
issues  to  be  considered  in  a  TOD  as  input  for  the  TOA  are 
addressed  by  the  database  design  and  algorithms  of  the  OSBATS. 
Based  upon  the  data  collected  in  this  effort,  these  assumptions 
are  not  valid.  For  example,  assessing  the  technological 
feasibility  of  device  design  options  is  an  important  part  of  the 
TOD  process  which  is  not  reflected  in  the  OSBATS.  Similarly, 


37 


defining  the  minimum  fidelity  requirements  necessary  to  train  a 
task  is  outside  the  scope  of  OSBATS. 

The  relationship  between  a  TOA  and  a  CTEA  within  the  OSBATS 
is  unclear.  However,  as  discussed  earlier  in  this  report,  the 
relationships  between  these  two  types  of  analyses  is  unclear 
generally. 

Comments  by  a  senior  engineer  familiar  with  OSBATS  and  the 
BCE  clearly  indicate  that  the  procedures  used  to  develop  cost 
estimates  within  the  OSBATS  do  not  match  the  rigorous  procedures 
used  in  developing  a  BCE.  Further,  the  OSBATS  is  not  designed  to 
support  the  job  of  developing  a  BCE. 

Developing  a  BTA  involves  consideration  of  detailed 
functional  specifications  which  may  go  beyond  the  level  of  detail 
addressed  by  the  OSBATS.  For  example,  an  engineer  must  consider 
hardware  and  software  variables  which  can  influence  the  ability 
of  a  device  to  collect,  organize  and  analyze  performance  data  to 
provide  feedback  to  students. 

The  Goals  of  the  OSBATS  and  CFP 

Examining  the  fit  between  the  goal  of  the  OSBATS  described  in 
the  first  paragraph  above  and  CFP  as  practiced  is  also  provides 
input  for  the  refinement  of  the  OSBATS.  Given  the  desire  of 
engineers  to  provide  an  effective  product  at  the  lowest  cost 
possible,  engineers  would  undoubtedly  feel  comfortable  with  the 
goal  of  OSBATS  (i.e.,  to  specify  the  design  and  use  of  training 
equipment  to  satisfy  training  requirements  at  a  minimum  cost,  or 
to  provide  the  greatest  training  effectiveness  for  a  given  cost) . 
However,  the  goal  of  the  OSBATS  does  not  necessarily  fit  the 
goals  of  all  schools  in  training  device  design.  In  cases  where  a 
TRADOC  school  is  prematurely  committed  to  a  particular  device, 
the  school's  goal  is  something  other  than  finding  the  most  cost- 
effective  training  device  option. 

Subaoals  and  Assumptions  on  Which  the  OSBATS  is  Based 

To  further  assess  the  fit  between  the  OSBATS  and  CFP 
practices  it  is  necessary  to  examine  the  subgoals  of  the  OSBATS 
and  the  manner  in  which  OSBATS  attempts  to  address  these 
subgoals.  As  described  later,  the  manner  in  which  the  OSBATS 
defines  an  "effective"  training  device  is  not  entirely  compatible 
with  the  engineers'  definition  of  "effective".  Further,  the 
OSBATS  model  does  not  allow  for  easy  insertion  of  expanded 
definitions  of  "effective." 

The  goal  of  optimizing  training  device  design  was  decomposed 
into  three  subgoals  (Sticha,  Blacksten,  et  al.,  1990); 
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1)  identify  tasks  that  are  good  candidates  for  training  that 
uses  a  training  device, 

2)  design  training  devices  with  a  level  of  sophistication 
and  cost  appropriate  for  the  tasks  that  the  devices  are 
to  train,  and 

3)  minimize  costs  through  the  appropriate  allocation  of 
training  resources  among  training  devices  and  actual 
equipment. 

As  mentioned  in  Appendix  C,  an  important  part  of  the  job  of 
engineers  is  to  work  with  schools  in  deciding  if  a  training 
device  is  needed  or  warranted.  Would  the  device  make  training 
more  cost-effective,  or  is  a  device  the  only  reasonable  way  to 
conduct  the  necessary  training  (e.g. ,  for  safety  reasons)?  Such 
questions  are  compatible  with  the  first  subgoal  listed  above. 

Once  the  need  for  a  device  is  established,  the  next  goal  of  the 
engineer  is  to  design  a  device  concept  that  addresses  the 
requirement  and  costs  less  than  other  alternatives  would  cost. 

The  engineer  devotes  his  or  her  energies  towards  attaining  this 
goal  throughout  the  remainder  of  the  CFP.  This  goal  is 
compatible  with  the  second  two  subgoals  listed  above. 

Implementation  of  Subaoals 

The  development  of  OSBATS  involved  breaking  subgoals  down 
into  functions,  breaking  functions  down  into  lower  level 
functions,  and  continuing  this  process  to  provide  the  level  of 
detail  needed  to  implement  a  software  demonstration  of  the 
OSBATS.  The  resulting  product  provided  tools  that  appear  to 
provide  engineers  with  the  capability  of  performing  portions  of 
the  CFP  better  and  faster;  however,  there  are  portions  of  the  CFP 
that  are  not  included  in  the  OSBATS. 

Software  modules  were  developed  to  implement  the  OSBATS 
models.  OSBATS  comprises  five  modules  that  address  a  spectrum  of 
critical  issues  that  confront  designers  of  training  devices.  The 
modules  assist  the  designer  in  performing  the  following  tasks; 

1)  cluster  tasks  to  allow  the  development  of  coherent 
training  device  configurations, 

2)  identify  optimal  instructional  features  for  each  task 
cluster, 

3)  specify  the  optimal  fidelity  levels  for  each  task 
cluster, 

4)  select  the  minimum  family  of  training  devices  that  meet 
the  training  requirements,  and 
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5)  optimize  allocation  of  training  resources  in  the  family 
of  suggested  training  devices  (Singer  and  Sticha,  1987) . 

The  OSBATS  does  not  address  all  of  the  issues  considered  by 
engineers  in  conducting  a  TOD  or  TOA,  and  it  does  not  provide  for 
incorporating  additional  issues.  Examples  of  these  issues 
include  consideration  of  the  technical  feasibility  of  device 
options  and  RAM  issues  associated  with  device  options.  The  case 
may  be  that  many  of  the  issues  that  are  not  included  in  the 
OSBATS  do  not  need  to  be  included.  This  is  a  substantial 
research  question  that  needs  to  be  addressed  as  the  OSBATS 
concept  is  integrated  with  the  overall  CFP  during  OSBATS 
refinement.  That  is,  the  current  documentation  for  OSBATS  does 
not  include  an  explanation  of  how  this  product  fits  into  the 
overall  CFP.  The  description  of  CFP  tasks  and  subtasks  provided 
in  Appendix  C  should  assist  in  preparing  such  documentation. 

As  noted  earlier,  there  are  at  least  five  factors  that  can 
influence  the  specific  procedures  used  in  training  device  CFP; 
whether  a  school  specifies  that  it  wants  a  specific  training 
device;  the  extent  to  which  the  project  involves  novel 
applications  of  training  technology;  whether  the  project 
addresses  individual  training  or  collective  training;  training 
device  policy  statements;  whether  the  project  addresses  school  or 
unit  training.  The  value  of  a  decision  support  system  should  be 
measured  in  terms  of  its  ability  to  accommodate  necessary 
procedural  diversity.  The  effort  to  develop  and  refine  the 
OSBATS  over  time  appears  to  be  attempting  to  address  procedural 
diversity,  as  described  below. 

The  developers  of  the  OSBATS  point  out  that  the  initial  model 
ij  in  :ended  to  apply  to  institutional  individual  skills  training 
(Sticha,  1990) ,  and  they  suggested  that  different  procedures 
might  be  required  for  collective  and  unit  training.  For  example, 
the  scope  of  OSBATS  was  limited  to  institutional  training, 
because  unit  training  considerations  were  judged  to  be  too 
complex  to  include  in  the  initial  development  of  OSBATS  (Sticha, 
Blacksten,  et  al.,  1990). 

The  OSBATS  accommodates  differences  among  projects  in  terms 
of  the  degree  to  which  a  school  specifies  a  particular  type  of 
device.  Although  the  OSBATS  is  capable  of  generating  large 
numbers  of  training  device  configurations  for  comparison  with  one 
another,  in  terms  of  cost-effectiveness,  the  OSBATS  also  allows 
the  engineer  to  specify  the  device  options  to  be  considered. 

This  same  flexibility  allows  the  engineer  to  select  device 
options  that  are  compatible  with  general  policy  statements. 
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Engineering  Tools  Within  the  OSBATS 

Any  attempt  to  describe  all  of  the  tools  within  the  OSBATS 
that  might  be  applied  to  address  existing  CFP  practices  better 
and  faster  would  be  a  substantial  project  in  its  own  right. 
Therefore,  this  chapter  will  highlight  one  of  the  most  powerful 
tools  within  OSBATS,  the  capability  to  compare  the  cost  and 
effectiveness  of  thousands  of  potential  design  options  within  a 
brief  period  of  time. 

OSBATS,  unlike  previous  aids  for  training  design,  is 
specifically  tailored  to  support  the  design,  rather  than  the 
evaluation,  of  training  devices  (Sticha,  Singer,  Blacksten, 
Morrison,  and  Cross,  1990).  Although  evaluation  models  can  be 
applied  to  device  design,  they  do  not  allow  convenient  comparison 
of  many  different  device  configurations.  That  is,  if  a  change  is 
made  in  a  device  configuration,  the  evaluation  process  must  be 
reinitiated  from  scratch.  OSBATS,  on  the  other  hand,  employs 
databases  and  algorithms  to  compare  device  configuration 
alternatives.  The  only  change  necessary  to  compare  an  additional 
alternative  is  a  change  in  a  portion  of  the  database  contents 
employed  in  the  comparison,  and  thus  OSBATS  makes  it  possible  to 
compare  a  new  configuration  in  a  few  minutes. 

The  OSBATS  is  capable  of  quickly  computing  changes  in  cost- 
effectiveness  in  response  to  changes  in  device  configuration 
because  it  encompasses  models  of  skill  acquisition,  retention, 
and  transfer.  Assumptions  concerning  the  relation  of  device 
fidelity  to  training  effectiveness  play  a  key  role  in  OSBATS. 
Also,  assumptions  concerning  the  contribution  of  instructional 
support  features  to  training  device  efficiency  are  critical  to 
the  OSBATS  model.  To  perform  cost/benefit  trade-off  analysis 
OSBATS  estimates  the  costs  of  combinations  of  fidelity  level  and 
instructional  feature  suites. 

The  potential  applications  of  the  OSBATS  tool  described  above 
go  beyond  a  TOA.  The  capability  to  compute  quickly  the 
effects  of  changes  in  device  configuration  on  the  relative 
cost-effectiveness  of  the  device  might  be  applied  to  developing 
the  BTA  and  subsequent  configuration  management  of  the  device 
design.  Once  again,  this  tool  is  only  one  of  the  tools  within 
OSBATS  that  might  be  applied  to  device  design.  Others  include 
an  algorithm  which  organizes  task  into  clusters  based  on 
similarities  in  their  "indications  for  simulation",  based  on 
consideration  of  multiple  factors  which  determine  the  value  of 
simulation-based  training. 

Conclusions 


•  The  OSBATS  was  designed  to  support  a  TOA.  It  was  not 

designed  to  support  two  important  events  which  lead  up  to 
the  TOA;  the  communication  between  TRADOC  schools  and  PM 
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TRADE  regarding  the  training  requirements  to  be  addressed 
by  a  device,  or  the  conduct  of  a  TOD.  The  relationship 
between  the  TOA  and  CTEA  within  the  scope  of  OSBATS  is 
not  clear,  but  the  relationship  of  these  analyses  is  not 
clear  outside  the  scope  of  OSBATS  either.  Similarly,  the 
role  of  the  OSBATS  in  developing  the  BTA  is  not  clear. 
However,  engineers  make  decisions  in  developing  the  BTA 
which  go  beyond  the  functions  of  the  OSBATS. 

•  OSBATS  does  not  address  all  issues  considered  by 
engineers  in  the  training  device  CFP.  This  finding 

is  difficult  to  interpret,  because  it  is  not  clear  as  to 
how  the  OSBATS  is  intended  to  be  integrated  with  the 
overall  CFP. 

•  The  prototype  OSBATS  does  not  address  embedded  training, 
collective  training,  or  training  in  units.  Therefore,  it 
applies  to  only  a  portion  of  the  CFP  projects  within  PM 
TRADE. 

•  The  OSBATS  contains  at  least  one  powerful  algorithm 
with  potential  applications  to  conducting  a  TOA, 
developing  a  BTA  and  managing  training  device 
configuration.  This  algorithm  might  help  to  link  these 
analyses  to  one  another  by  providing  a  common  method 
for  comparing  the  cost~effectiveness  of  device 
configurations. 

Recommendations 

•  Ongoing  efforts  to  refine  the  OSBATS  should  document  how 
the  OSBATS  is  intended  to  fit  in  the  overall  training 
device  CFP  described  in  Appendix  C. 

•  At  a  minimum,  the  OSBATS  should  be  designed  to  provide 
greater  flexibility  in  terms  of  incorporating  information 
developed  during  the  TOD  which  needs  to  be  included  in  a 
TOA. 

•  Ongoing  efforts  to  refine  the  OSBATS  should  document  if 
and  how  this  aid  will  accommodate  procedural  diversity 
influenced  by:  the  extent  to  which  a  school  specifies 
that  it  wants  a  particular  training  device;  the  extent  to 
which  the  project  involves  novel  applications  of  training 
technology;  whether  the  project  addresses  individual 
training  or  collective  training;  training  device  policy 
statements;  whether  the  project  addresses  school  or 

unit  training. 
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•  PM  TRADE  should  consider  application  of  selected  OSBATS 
features  as  a  tool  in  developing  the  BTA  and  managing 
device  configuration. 
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Summary 


From  a  top  level  perspective,  the  CFP  as  practiced  tends  to 
match  the  CFP  described  by  regulations  and  other  guidance  with 
two  exceptions.  First,  engineers  play  a  more  active  role  in 
defining  training  requirements  than  is  implied  in  regulations. 
Second,  engineers  conduct  a  preliminary  TOA  to  evaluate  the 
quality  of  their  TOD,  rather  than  waiting  for  schools  to  initiate 
the  TOA. 

CFP  guidance  is  written  at  a  general  level  and  does  not 
discuss  in  much  detail  the  relationships  among  a  TOD,  a  TOA,  a 
CTEA,  the  BTA,  and  device  configuration  management.  Questions 
were  raised  regarding  the  need  for  conducting  a  TOA  separate  from 
a  CTEA,  and  a  question  was  raised  regarding  the  continuity 
between  the  BTA  and  device  configuration  management. 

From  a  lower  level  perspective,  there  are  at  least  five 
factors  that  influence  the  specific  procedures  used  by  engineers 
during  the  training  device  CFP;  whether  a  school  specifies  that 
it  wants  a  specific  training  device;  the  extent  to  which  the 
project  involves  novel  applications  of  training  technology; 
whether  the  project  addresses  individual  training  or  collective 
training;  training  device  policy  statements;  whether  the  project 
addresses  school  or  unit  training.  The  effects  that  each  of 
these  factors  have  on  CFP  procedures  need  to  be  examined  in 
greater  detail. 

Engineers  consider  the  effects  that  principles  of  learning 
have  on  materiel  solutions  to  training  requirements.  The  extent 
to  which  these  principles  are  considered  is  not  standardized, 
except  that  all  engineers  appear  to  be  concerned  with  defining 
the  minimum  fidelity  requirements  necessary  to  support  training. 
Differences  among  projects  in  terms  of  the  learning  principles 
considered  appear  to  be  a  function  of  the  engineer  (some  are  more 
aware  of  learning  principles)  and  the  learning  issues  or  training 
technology  issues  unique  to  projects.  To  insure  the  principles 
of  learning  are  considered  along  with  other  training  variables, 
there  is  a  need  to  develop  procedures  which  link  engineering 
variables  with  learning  variables. 

Engineers  are  generally  unhappy  about  the  amount  and  quality 
of  information  that  schools  provide  regarding  the  training 
requirements  to  be  addressed  by  a  device.  The  most  critical  gaps 
in  information  are  a  failure  to  describe  the  tasks  to  be  trained, 
and  a  failure  to  describe  the  minimum  fidelity  requirements. 
Additional  gaps  in  the  information  provided  by  schools  are 
dependent  upon  such  factors  as  the  type  or  scope  of  training 
requirements  to  be  addressed.  Present  guidance  for  use  by 
schools  in  preparing  training  requirements  is  of  the  "one  size 
fits  all”  variety.  There  is  a  need  to  inform  schools  of  engineer 
information  needs  associated  with  various  types  of  training 
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device  projects,  but  PM  TRADE  and  ARI  must  first  identify  these 
information  requirements. 

The  prototype  OSBATS  decision  support  system  was  designed  to 
address  the  TOA  in  isolation,  and  it  does  not  consider  the 
relationships  which  the  TOA  has  with  describing  training 
requirements,  performing  a  TOD,  conducting  a  CTEA,  developing  a 
BTA  and  developing  a  BCE.  One  major  problem  resulting  from  this 
lack  of  consideration  is  that  the  OSBATS  is  unable  to  accommodate 
all  of  the  types  of  issues  addressed  by  engineers  in  performing  a 
TOD.  Another  problem  is  that  the  OSBATS  fails  to  address  the 
most  substantial  problem  faced  by  engineers  during  the  CFP, 
inadequate  information  regarding  the  tasks  to  be  trained  by  a 
device. 

Ongoing  efforts  to  refine  the  OSBATS  must  consider  that  the 
TOA  is  not  an  isolated  part  of  the  CFP.  The  documentation  for 
the  OSBATS  should  clearly  explain  how  this  tool  is  to  be 
integrated  with  other  portions  of  the  CFP,  and  the  OSBATS  should 
be  designed  to  support  this  integration.  For  example,  if  the 
OSBATS  is  intended  to  support  the  conduct  of  a  TOA,  then  it 
should  be  able  to  handle  all  of  the  types  of  information 
generated  during  a  TOD. 

Specific  targets  of  opportunity  for  improving  CFP  are 
provided  below.  It  is  recommended  that  an  assessment  be  made  of 
the  progress  made  in  addressing  these  targets  at  a  later  point  in 
time. 

•  Standardize  the  extent  to  which  engineers  consider 
learning  principles  in  their  work  in  a  manner  that 
takes  into  account  important  differences  among  projects 
(e.g.,  individual  versus  collective  training). 

•  Insure  that  schools  are  informed  of  the  manner  in  which 
information  needs  of  engineers  vary  as  a  function  of 
projects. 

•  Clarify  the  relationships  among  a  TOA,  a  CTEA,  a  BTA 
and  device  configuration  management. 

•  Provide  engineers  with  easier  access  to  data  on  training 
requirements,  training  technology,  skill  acquisition/ 
retention,  training  environments,  and  costs. 

•  Develop  procedures  for  integrating  training  device 
projects. 

•  Define  the  scope  of  activities  to  be  addressed  by  a 
refined  OSBATS,  and  explain  how  the  refined  OSBATS  is 
to  be  integrated  with  any  CFP  activities  outside  its 
scope. 


46 


References 


Armstrong,  J.N.,  &  Deaver,  R.E.  (1990).  ’'Traini’.g  in  the  21st 
century .. .The  Army's  long  range  training  plan."  Army 
Research.  Development,  and  Acquisition  Bulletin.  PB  70-90- 
1,  6-7.  Alexandria,  VA:  U.S.  Army  Materiel  Command. 

Chenzoff ,  A.P. ,  Joyce,  R.P.  &  Nauta,  F.  (1985).  Maintenance  job 
aids  in  the  U.S.  Navy;  Present  status  and  future  directions. 
Orlando,  FL:  Naval  Training  Equipment  Center. 

Department  of  the  Army  (1986).  Systems  acquisition  policy  and 

procedures  (Army  Regulation  70-1).  Washington,  D.C.:  Author. 

Department  of  the  Army  (1987).  Materiel  objectives  and 

recTuirements  (Army  Regulation  71-9).  Washington,  D.C.: 
Author. 

Ditzian,  J.L.,  Roth,  J.T.,  &  Johnston,  E.  (1987).  Design 
specification  for  a  MANPRINT  Training  Characteristics 
Estimation  Aid  (TCEAl .  Butler,  PA:  Applied  Science 
Associates,  Inc. 

Finley,  D.L. ,  Alderman,  I.N.,  Peckham,  D.S.,  &  Strasel,  H.C. 
(1988).  Implementing  Embedded  Training  fET> :  Volume  1  of 
10:  Overview  (ARI  Research  Product  88-12).  Alexandria,  VA; 
U.S.  Army  Research  Institute  for  the  Behavioral  and  Social 
Sciences.  (AD  A201  401) 

Heeringa,  L.M. ,  Baum,  p.R.,  Holman,  G.L.  &  Peio,  K.J.  (1982). 

An  analysis  of  training  device  reguirements  documentation 
procedures  (SIMTRAIN  Task  1.4  Final  Report,  Contract  Number 
MDA  903-81-C-0214) .  Minneapolis,  MN:  Honeywell  Systems  and 
Research  Center. 

Hinton,  W.M.,  Braby,  R. ,  Feuge,  R.L. ,  Stultz,  A.H.,  Evans,  S.M., 
Gibson,  M.R.,  &  Zaldo,  W.T.  (1990).  A  design  architecture 
for  an  integrated  training  system  decision  support  system 
(ARI  Research  Report  1566).  Alexandria,  VA:  U.S.  Army 
Research  Institute  for  the  Behavioral  and  Social  Sciences. 
(AD  A225  084) 

Post,  T.J.  &  Price,  H.E.  (1973).  Development  of  optimum 

performance  aids  for  troubleshooting.  Falls  Church,  VA: 
BioTechnology,  Inc. 

Riedel,  S.L.  (1988).  User  acceptance  and  field  implementation  of 
decision  support  systems  (  ARI  Research  Report  1477). 
Alexandria,  VA:  U.S.  Army  Institute  for  the  Behavioral  and 
Social  Sciences.  (AD  A200  412) 


47 


Singer,  M.J.  &  Sticha,  P.J.  (1987,  November) .  Designing 

training  devices;  The  optimization  of  simulation-based 
training  systems.  Paper  presented  at  the  Ninth 
Interservice/Industry  Training  Conference,  Washington  DC. 

Sticha,  P.J.  (1990) .  Research  and  development  on  the 

characterization  of  simulation-based  training  system s.:_ 
Project  executive  summary  (ARI  Technical  Rep.jrt  9C:;. 
Alexandria,  VA:  U.S.  Army  Institute  for  the  Behavioral  and 
Social  Sciences.  (AD  A227  361) 

Sticha,  P.J.,  Blacksten,  H.R.,  Buede,  D.M. ,  Sing#*r,  M.J., 
Gilligan,  E.L. ,  Mumaw,  R.J.  &  Morrison,  O.E.  (1990). 
Optimization  of  simulation-based  training  systems;  Model 
description,  implementation  and  evaluation  (ARI  Technical 
Report  896) .  Alexandria,  VA:  U.S.  Army  Institute  for  the 
Behavioral  and  Social  Sciences.  (AD  A237  266) 

Sticha,  P.J.,  Singer,  M.J.,  Blacksten,  H.R.,  Morrison,  J.E. ,  & 
Cross,  K.D.  (1990) .  Research  and  Methods  for  Simulation 
Design:  State  of  the  Art  (ARI  Technical  Report  914) . 
Alexandria,  VA:  U.S.  Army  Institute  for  the  Behavioral  and 
Social  Sciences.  (AD  A230  076) 

Strasel,  H.C.,  Dyer,  F.N. ,  Roth,  J.T. ,  Alderman,  I.N.  & 

Finley,  D.L.  (1988).  Implementing  Embedded  Training  TET) : 
Volume  2  of  10:  Embedded  Training  as  a  system  alternative 
(ARI  Research  Product  88-22).  Alexandria,  VA:  U.  S.  Army 
Research  Institute  for  the  Behavioral  and  Social  Sciences. 

(AD  A204  836) 

U.S.  Army  Project  Manager  for  Training  Devices  (1986).  Project 
management  and  execution  (Standard  Operation  Procedure  66) . 
Orlando,  FL;  Author. 

U.S,  Army  Training  Support  Center  Devices  Management  Directorate 
(1988).  Training  developer's  procedural  guide:  Training 
device  strategies.  Fort  Eustis,  VA:  Author. 

U.S.  Army  Training  Support  Center  Devices  Management  Directorate. 
(1989a).  Training  developer’s  procedural  guide:  Nonsvstem 
training  device  reguirements  documentation.  Fort  Eustis,  VA; 
Author. 

U.S.  Army  Training  Support  Center  Devices  Management  Directorate 
(1989b).  Training  developer's  procedural  guide;  System 
training  device  reguirements  documentation.  Fort  Eustis,  VA; 
Author. 

U.S.  Army  Materiel  Command  and  U.S.  Army  Training  and  Doctrine 
Command  (1987).  Materiel  acouisition  handbook  (Pam  70-2). 
Alexandria,  VA  and  Fort  Monroe,  va;  Authors. 


48 


U.S.  Army  Materiel  Command  and  U.S.  Army  Training  and  Doctrine 
Command  (1988) .  Concept  Formulation  Process  fCFP)  Memorandum 
of  Instruction  fMOI) .  Alexandria,  VA  and  Fort  Monroe,  VA: 
Authors . 


49 


APPENDIX  A 

ENGINEERING  FUNCTIONS  SURVEY 


Please  look  over  the  entire  questionnaire  before  you  begin  your 
answers.  If  you  need  additional  space  to  complete  an  answer  please 
continue  on  the  back  of  the  page. 

If  you  have  any  questions  about  the  purpose  of  this  questionnaire, 
or  specific  items,  please  call  Larry  Meliza  (4374)  or  Don  Lampton 
(4368)  . 

A.  BACKGROUND  INFORMATION 


1.  Education  (Please  indicate  your  major  or  majors  for 
each  degree) 


Some  College  _ 

B.S  _ 

Graduate  Degree (s) _ 

2.  How  long  have  you  worked  for  PM  TRADE? 


3.  How  long  have  you  worked  in  E  Division? _ 

4.  How  many  years  experience  do  you  have  in  designing 

training  devices? _ 


5.  Place  a  check  mark  in  front  of  each  type  of  device 
(or  device  component)  on  which  you  have  worked 


Maintenance  Trainers 
Armor  Trainers 
Gunnery  Trainers 
Procedural  Trainers 
Computer  Based  Devices 
Full  Mission  Simulators 
Weapons  Effects  Devices 

Flat  Panel  Trainers 
Collective/Team  Trainers 
Motion  Simulation 
Other  (Please  list  below) 


Aviation  Trainers 
Driver  Trainers 
Operations  Trainers 
Command  and  Control  Trainers 
Part  Task  Trainers 
Low  Fidelity  Devices 
Troubleshooting  Training 
Devices 

Electronics  Maintenance 
Instructor/Operator  Station 
Interactive  Threat  Force 
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6.  Please  indicate  the  amount  of  experience  you  have  had  with 
each  category  of  activity  or  document  listed  below. 

No  One  Two  or  More 


Experience 

Project 

Projects 

a. 

Training  device  acquisition 

(  ) 

{  ) 

( 

) 

b. 

Weapon  system  acquisition 

(  ) 

(  ) 

( 

) 

c. 

Non-developmental  items 

(  ) 

(  ) 

( 

) 

d. 

Training  Device  Requirements 
(TDRs) 

(  ) 

(  ) 

( 

) 

e. 

Commercial  Training  Device 
Requirements  (CTDR) 

(  ) 

(  ) 

( 

) 

f . 

Systems  Training  Plan  (STRAP) 

(  ) 

{  ) 

( 

) 

g- 

Systems  MANPRINT 

Management  Plan  (SMMP) 

(  ) 

(  ) 

( 

) 

h. 

Training  Development 

Study  (TDS) 

(  ) 

(  ) 

( 

) 

i . 

Concept  Formulation 

Package  (CFP) 

(  ) 

(  ) 

( 

) 

j  • 

Trade  Off  Determination  (TOD) 

(  ) 

(  ) 

( 

) 

k. 

Trade  Off  Analysis  (TOA) 

(  ) 

(  ) 

( 

) 

1. 

Cost  and  Training 

Effectiveness  Analysis  (CTEA) 

(  ) 

(  ) 

( 

) 

m. 

Best  Technical  Approach  (BTA) 

(  ) 

(  ) 

( 

) 

n. 

Embedded  Training 

(  ) 

(  ) 

( 

) 

o. 

Collective/Team  Training 

(  ) 

(  ) 

( 

) 

P- 

Broad  Agency  Announcement 
(BAA)  proposals 

(  ) 

(  ) 

( 

) 

7.  Which  Army  schools,  and  other  institutions,  have  you  worked  with 
in  designing  training  devices  during  the  last  two  years? 

8.  Place  a  check  mark  in  front  of  the  formal  reviews  for  which 
you  have  helped  to  prepare  input. 


JWG  1 

MARB  I 

JWG  2 

MARB  II 

AMC  MILESTONE 

AMC  MILESTONE 


I/II  Position 
I/III  Position 


MILESTONE  I/II  I PR 
MILESTONE  I/III  I PR 
Quarterly  Reviews 
Monthly  Project  Screening 
Committee  (PSC) 


9.  What  training  have  you  had  in  the  field  of  systems  engineering? 
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10.  A  list  of  documents  which  other  Army  organizations  are 
responsible  for  producing  is  provided  below.  Let  us  know 
whether  you  helped  other  organizations  to  prepare  these 
documents,  conducted  a  formal  review  of  these  products  or 
use  these  products  in  your  work.  Mark  as  many  situations  as 
apply  for  each  document,  and  add  any  documents  which  have 
been  left  out. 


Help 

Other 

Organizations 
to  Prepare 


Review 
and  Provide 
Feedback 


Use  in  My 
Work 


a. 

Systems  MANPRINT 

( 

) 

( 

) 

( 

) 

b. 

Management  Plan  (SMMP) 
Operational  and 
Organizational  (O&O)  Plan 

( 

) 

( 

) 

( 

) 

c. 

Systems  Training  Plan 

( 

) 

( 

) 

( 

) 

d . 

(STRAP) 

Training  Device 

( 

) 

( 

) 

( 

) 

e. 

Requirement  (TDR) 

Commercial  Training  Device 

( 

) 

( 

) 

( 

) 

f . 

Requirement  (CTDR) 

Trade  Off  Analysis 

( 

) 

( 

) 

( 

) 

g- 

(TOA) 

Requirements  for 

( 

) 

( 

) 

( 

) 

h. 

Operational  Capability  (ROC) 

( 

) 

( 

) 

( 

) 

i. 

( 

) 

( 

) 

( 

) 

11. 

We  would  like  to  know  three 

things 

about  each 

type 

of  decision 

listed  below.  First,  do  you  make  this  decision  in  your  work  on 
either  a  formal  or  informal  basis?  Second,  are  there  problems  in 
making  the  decision  due  to  a  lack  of  decision  rules  or 
information.  Third,  were  you  able  to  overcome  the  problem  you 
faced,  or  do  you  feel  you  have  considerable  knowledge  about  why  a 
particular  decision  might  be  difficult  to  address. 


If  there  are  other  training  decisions  to  be  made  which  are  not 
listed  below,  please  add  them  to  the  list. 


DO  YOU  HELP  DECIDE: 

DECISION 

PROBLEM 

OVERCOME/ 

a.  If  a  proposed  training  device 
would  duplicate  an  existing 

MADE 

FACED 

KNOWLEDGE 

device  (or  device 
under  development)? 

(  ) 

(  ) 

(  ) 
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b.  If  fidelity  requirements  (e.g. 

degrees  of  motion  for  a  flight  (  )  (  )  {  ) 

simulator)  are  clearly  specified? 

c.  If  fidelity  requirements  are  (  )  (  )  (  ) 

valid? 

d.  How  much  it  will  cost  to  include  (  )  (  )  (  ) 

a  particular  instructional  feature 

in  a  training  device? 

f.  Which  instructional  support 

features  should  be  included  in  a  (  )  (  )  (  ) 

simulator  or  embedded  training 

application 

g.  Which  training  device  options 
(e.g.,  full  mission  simulator, 

embedded  training)  or  combination  (  )  (  )  (  ) 

of  options  are  the  most/least 

cost-effective 

i.  Whether  a  particular  training  (  )  (  )  (  ) 

device  is  compatible  with  the 
institutional  and/or  unit  training 
environment 

k.  If  a  Trade  Off  Deteirmination  (TOD)  (  )  (  )  (  ) 

adequately  defines  the  issues  to 
be  addressed  in  a  Trade  Off 
Analysis  (TOA) 

12.  Have  there  been  any  situations  where  you  thought  a  major 
error  was  being  made  in  the  design  of  a  training  device? 

If  so,  were  you  able  to  do  anything  to  correct  the  problem? 


13.  What  software  do  you  use  in  your  work? 
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14.  What  databases  do  you  use  in  your  work? 


B.  ORGANIZATIONAL  AND  INDIVIDUAL  FUNCTIONS 

1.  List  the  most  important  functions  performed  by  E  Division, 
and  place  a  checkmark  in  front  of  the  functions  performed 
by  your  branch. 


2.  Please  list  your  general  job  functions,  and  list  the  major 
tasks  you  perform  within  each  function.  (Note:  Certain  tasks  may 
apply  to  more  than  one  function.) 


3.  For  each  function  you  reported  in  item  2,  describe  the  input 
you  receive  and  the  expected  output. 
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4.  For  each  function  you  reported  in  item  2,  list  the  sources  of 
information  you  use  in  addressing  the  function.  Include  any 
guidelines  or  checklists  you  have  developed  as  well  as  people 
resources  (e.g.,  contractors,  HEL,  school  DOTD  staff). 


TRAINING  AND  LEARNING  CONSIDERATIONS  IN  DEVICE  DESIGN 

1.  What  types  of  training  or  learning  issues  do  you  consider 
in  your  work? 


2.  Please  list  what  you  consider  to  be  the  significant  steps  or 
decision  points  in  arriving  at  the  Best  Technical  Approach  (BTA) . 


3.  What  are  the  most  significant  training  decisions  that  bear 
upon  the  BTA? 
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CAREER  DEVELOPMENT  AND  JOB  SATISFACTION 

1.  What  are  the  najor  obstacles  to  performing  your  job 
efficiently? 


2 .  What  information  would  assist  you  in  better  performing 
your  job  duties? 


3.  Of  the  tasks  that  you  must  perform  in  your  job,  which  would 
you  most  like  for  someone  else  to  perform?  Why? 


4.  Of  the  decisions  that  you  must  make  in  your  work,  which  would 
you  most  like  to  have  someone  else  make? 
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5.  Which  parts  of  your  job  give  you  the  greatest  satisfaction? 


6.  What  training  would  you  like  to  receive  that  you  feel 
would  help  you  to  better  perform  your  duties? 
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APPENDIX  B 

EXPERIENCE  OF  SURVEY  PARTICIPANTS 


Education.  All  of  the  engineers  have  at  least  a  Bachelors 
Degree.  Over  half  have  a  master's  degree,  and  two  have 
doctorates.  Roughly  three  quarters  of  the  engineers  have  degrees 
in  electrical  engineering,  and  most  of  the  rest  have  degrees  in 
either  math,  computer  science  and  physics. 

Years  of  Experience  within  E  Division.  The  number  of  years  of 
experience  within  E  Division  for  individuals  involved  in  concept 
formulation  ranged  from  nine  months  to  fourteen  years  with  a 
median  of  four  years  of  experience.  The  four  engineers  from  ET 
Branch  had  worked  for  E  Division  for  one,  three  and  one-half, 
seven  and  eight  years  respectively. 

Amount  and  Variety  of  Experience  in  Device  Design.  The  number 
of  years  of  experience  in  the  design  of  training  devices  ranged 
from  one  year  to  22  years  with  a  median  of  six  years.  Table  B-1 
illustrates  the  variety  of  experience  ED  and  EC  engineers 
in  terms  of  the  types  of  devices  on  which  they  have  worked. 

Table  B-2  illustrates  the  breadth  of  user  groups  with  which 
E  Division  engineers  work.  Most  engineers  had  worked  with 
more  than  one  school  during  the  two  year  sample  period. 

TABLE  B-1.  NUMBER  OF  ENGINEERS  REPORTING  EXPERIENCE  WORKING  WITH 
VARIOUS  TYPES  OF  TRAINING  DEVICES 

TYPES  OF  TRAINING  DEVICES  NUMBER  OF  ENGINEERS  REPORTING 

EXPERIENCE  WORKING  WITH  DEVICE 
TYPE 


COMPUTER-BASED  DEVICES  17 
COLLECTIVE/TEAM  TRAINERS  12 
INSTRUCTOR  OPERATOR  STATION  12 
PROCEDURAL  TRAINERS  10 
PART  TASK  TRAINERS  9 
MAINTENANCE  TRAINERS  9 
OPERATIONS  TRAINERS  9 
GUNNERY  TRAINERS  9 
FULL  MISSION  SIMULATORS  9 
COMMAND  AND  CONTROL  TRAINERS  8 
WEAPONS  EFFECTS  DEVICES  8 
ARMOR  TRAINERS  7 
ELECTRONICS  MAINTENANCE  7 
DRIVER  TRAINERS  6 
FLAT  PANEL  TRAINERS  5 
MOTION  SIMULATION  5 
INTERACTIVE  THREAT  FORCE  5 
LOW  FIDELITY  DEVICES  4 


TROUBLESHOOTING  TRAINING  DEVICES  4 
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TABLE  B-2.  NUMBER  OF  E  DIVISION  ENGINEERS  WHO  HAVE  WORKED  WITH 
SELECTED  SCHOOLS  AND  CENTERS  DURING  THE  PAST  TWO  YEARS 


SCHOOL/ CENTER 


NUMBER  OF  ENGINEERS  WORKING 
WITH  EACH  SCHOOL  OR  CENTER 


AIR  TRAFFIC  CONTROL  SCHOOL  1 
ARMOR  SCHOOL  6 
AVIATION  SCHOOL  6 
AIR  DEFENSE  ARTILLERY  SCHOOL  3 
CHEMICAL  SCHOOL  2 
COMBINED  ARMS  TRAINING  ACTIVITY  7 
COMBAT  TRAINING  CENTERS  4 
COMMAND  AND  GENERAL  STAFF  COLLEGE  1 
ENGINEER  SCHOOL  2 
FIELD  ARTILLERY  3 
INFANTRY  SCHOOL  3 
INTELLIGENCE  SCHOOL  1 
INTELLIGENCE  SCHOOL  AND  CENTER  4 
LOGISTICS  CENTER  2 
SIGNAL  SCHOOL  1 
TRANSPORTATION  SCHOOL  2 


TABLE  B-3  indicates  the  number  of  engineers  with  experience 
preparing  input  for  the  various  reviews  and  decision  meetings 
included  in  CFP.  Tables  B-4  and  B-5  indicate  engineer  experience 
with  various  documents,  activities  and  content  areas  in  CFP. 

TABLE  B-3.  NUMBER  OF  ENGINEERS  WITH  EXPERIENCE  PREPARING  INPUT 
FOR  THE  VARIOUS  TYPES  OF  DECISION  MEETINGS  AND  REVIEWS  WITHIN 
TRAINING  DEVICE  CFP 

REVIEW  OR  NUMBER  OF  ENGINEERS  WHICH 

DECISION  MEETING  HAVE  HELPED  TO  PREPARE  INPUT 

FOR  REVIEW 


JWGl  15 

MARBl  13 

JWG2  10 

MARB2  12 

AMC  MILESTONE  I/II  POSITION  7 

AMC  MILESTONE  I/I I I  POSITION  4 

MILESTONE  I/II  IPR  10 

MILESTONE  1/ III  IPR  7 

QUARTERLY  REVIEWS  12 

MONTHLY  PROJECT  SCREENING  COMMITTEE  11 
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TABLE  B-4.  NUMBER  OF  ENGINEERS  WITH  EXPERIENCE  PREPARING, 
REVIEWING  AND  USING  SELECTED  CFP  DOCUMENTS 


DOCUMENT  HELP 

REVIEW 

USE  IN  ] 

OTHER 

AND  PROVIDE 

WORK 

ORGANIZATIONS 

FEEDBACK 

TO 

PREPARE 

SYSTEMS  MANPRINT 

5 

12 

5 

MANAGEMENT  PLAN  (SMMP) 
OPERATIONAL  AND 
ORGANIZATIONAL  (O&O)  PLAN 

4 

6 

8 

SYSTEMS  TRAINING  PLAN 

1 

3 

3 

(STRAP) 

TRAINING  DEVICE 

10 

15 

15 

REQUIREMENT  (TDR) 
COMMERCIAL  TRAINING 

DEVICE  REQUIREMENT  (CTDR) 

9 

11 

11 

TRADE  OFF  ANALYSIS 

9 

12 

11 

REQUIREMENTS  FOR 
OPERATIONAL  CAPABILITY 

3 

10 

11 

TABLE  B-5.  NUMBER  OF  ENGINEERS  WITH  VARIOUS  DEGREES  OF  EXPERIENCE 
WITH  SELECTED  TYPES  OF  CFP  DOCUMENTS  AND  ACTIVITIES 


ACTIVITY  OR  DOCUMENT 


NO  ONE  TWO  OR  MORE 

EXPERIENCE  PROJECT  PROJECTS 


TRAINING  DEVICE  ACQUISITION  2  3 

WEAPON  SYSTEM  ACQUISITION  9  0 

NON-DEVELOPMENTAL  ITEMS  5  5 

TRAINING  DEVICE  REQUIREMENTS 
(TDRS) 

COMMERCIAL  TRAINING  DEVICE  5  4 

REQUIREMENTS  (CTDRS) 

SYSTEMS  TRAINING  PLAN  12  2 

SYSTEMS  MANPRINT  MANAGEMENT  5  4 

PLAN  (SMMP) 

TRAINING  DEVELOPMENT  STUDY  4  5 

CONCEPT  FORMULATION  PACKAGE  2  5 

TRADE  OFF  DETERMINATION  (TOD)  2  4 

TRADE  OFF  ANALYSIS  (TOA)  2  3 

COST  AND  TRAINING  6  4 

EFFECTIVENESS  ANALYSIS  (CTEA) 

BEST  TECHNICAL  APPROACH  (BTA)  1  5 

EMBEDDED  TRAINING  8  3 

COLLECTIVE/TEAM  TRAINING  5  4 

BROAD  AGENCY  ANNOUNCEMENT  4  5 

(BAA)  PROPOSALS 


13 

8 

8 


9 

3 

8 

6 

11 

11 

11 

7 

11 

5 

5 

8 
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APPENDIX  C 

E  DIVISION  TASKS  AND  SUBTASKS 
PERFORMED  DURING  THE  CONCEPT  FORMULATION  PROCESS 


The  major  tasks  performed  by  engineers  during  the  concept 
formulation  process  are  listed  below  with  a  brief  description  of 
the  purpose  of  each  task.  Subtasks  are  listed  for  each  task. 

These  descriptions  were  taken  from  survey  responses  and  responses 
to  interview  questions.  The  material  below  describes  the  range  of 
activities  performed  by  engineers,  and  it  does  not  necessarily 
describe  the  job  as  performed  by  every  engineer. 

The  first  two  tasks  described  below  go  beyond  E  Division's 
role  in  concept  formulation  as  specified  in  regulations, 
memorandum  of  instruction  and  SOPS.  These  tasks  are  concerned 
with  defining  training  requirements  in  adequate  detail  to 
develop  engineering  solutions.  In  an  effort  to  be  responsive 
to  user  requests,  E  Division  engineers  aggressively  seek  out 
the  information  required  from  these  users. 

Task;  Provide  Input  for  Some  or  all  of  the  Components  of  a 
Preliminary  Training  Development  Study  fPTDS)  and  Training 
Development  Need  Statement  for  a  Nonsvstem  Device  or  a  Preliminary 
Training  Effectiveness  Analysis  for  a  System  Device.  The  purpose 
of  this  task  is  to  provide  the  initial  definition  of  a  training 
requirement  that  might  be  addressed  by  a  training  device.  Further 
development  of  the  output  of  this  task  should  result  in  the 
Training  Device  Requirement  (TDR)  addressed  by  the  subsequent 
task. 

Subtasks: 


•  Insure  that  the  training  requirement  is  adequately 

described  to  guide  device  design 

-  Encourage  the  user  (e.g.,  schools  and  integrating 
centers)  to  define  the  intended  scope  of  the 
device (s)  in  terms  of  duty  positions  to  be  trained 
and  the  types  of  tasks  to  be  trained  (e.g.,  trouble 
shooting  electrical  problems) 

-  Insure  that  the  user  has  stated  an  appropriate  goal 
in  pursuing  the  development  of  a  training  device 
(e.g.,  reduce  training  costs,  provide  training 

on  new  equipment,  increase  the  effectiveness  of 
training) 

-  Encourage  the  user  to  conduct  job,  task  and  skill 
analyses  or  monitor  a  service  support  contractor  in 
performing  these  tasks 
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-  visit  the  school  to  find  out  more  about  the  tasks 
to  be  trained 

-  Insure  performance  standards  are  available  for  each  task 

-  Maintain  a  knowledge  of  school  specific  and  TRADOC- 
wide  training  strategies  and  needs  through  phone 
conversations,  site  visits,  attending  meetings  and 
reviewing  policy  documents  (e.g.,  white  papers). 

For  System  Devices 

-  Review  documents  describing  the  new  weapon  system, 
including;  draft  Operational  and  Organizational  (O&O) 
Plan,  System  Training  Plan  (STRAP) ,  concept  formulation 
papers  and  available  contractor  design  documents 

-  Talk  with  combat  developers  to  find  out  how  the  new 
weapon  system  is  expected  to  differ  from  related  or 
predecessor  weapon  systems  (e.g.,  new  computer-assisted 
navigation  system)  and  make  sure  these  differences  are 
reflected  in  the  job,  task  and  skill  analyses 

•  Determine  when  the  user  expects  to  field  the  device  (in 
six  months?  five  years?) 

•  Determine  if  and  what  the  funding  limits  are  for  a 
device  or  devices 

•  Conduct  Media  Analysis  or  review  results  of 
school/contractor  performed  media  analysis 

-  Identify  similar  projects  and  review  the  results  of 
the  media  analysis  for  these  projects 

-  Insure  that  fidelity  requirements  for  training 
each  task  are  specified  in  adequate  detail  to 
drive  device  design  (e.g.,  if  motion  is  required, 
are  the  ranges  specified?) 

-  Verify/negotiate  fidelity  requirements 

-  Make  sure  tasks  requiring  sustainment  training  and  hands 
on  training  have  been  identified 

-  Make  sure  that  the  results  of  the  analysis  specify  the 
location  of  training  (institution  or  unit) 

•  Verify/negotiate  training  device  alternatives  to 
be  addressed  by  a  Trade-Off  Determination  (TOD) 

-  Make  sure  that  alternative  device  concepts  are 
present 
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-  Insure  that  there  is  an  adequate  training  concept  or 
strategy  for  each  alternative,  and  make  sure  that 
the  strategy  explains  how  the  alternative  is  to  be 
used  in  the  total  training  context  (e.g.,  the  device 
will  be  used  after  classroom  instruction  on  the  topic 
and  before  training  is  conducted  on  a  full  mission 
simulator) 

-  Make  sure  that  the  alternatives  are  consistent  with 
the  results  of  a  media  analysis 

-  Make  sure  that  the  number  of  individuals  or  units 
to  be  trained  have  been  identified 

-  Insure  that  estimates  are  available  regarding  the 
number  of  devices  that  will  be  required 

Task;  Review  TDR  f f or  Nonsvstem  Devices)  or  Training  Device 
Requirement  Appendix  of  a  ROC  Document  (for  System  Devices). 

Subtasks: 

•  Insure  that  the  requirement  provides  all  of  the 
information  required  of  a  PTDS  or  PTEA  (see  above) 

•  For  system  devices,  review  updates  in  weapon  system 
design  or  in  planning  documents  (e.g. ,  O&O  Plan)  to 
make  sure  training  requirements  defined  in  the  PTEA 
have  not  changed  or  changes  have  been  reflected  in  a 
revised  PTEA  and  device  requir'^ment 

•  Review  task/skills  trained  by  existing  devices,  where 
appropriate  to  make  sure  that  there  is  not  unnecessary 
duplication  of  devices 

•  Decide  if  the  requirements  leave  engineers  with  the 
flexibility  to  design  cost-effective  solutions 

Task;  Review  Commercial  Training  Device  Recmirement  (CTDR)  in 
Cases  where  a  School  Believes  an  "Off  the  Shelf"  Product  Might  be 
Used  to  Address  Training  Requirements. 

Subtasks: 

•  Make  sure  that  the  CTDR  adequately  describes  the 
training  requirement  (see  above) 

•  Make  sure  that  the  device  can  be  procured  without 
research,  development,  test  and  evaluation. 

-  Obtain  and  review  available  documentation  on  the 
commercial  product  to  identify  tasks  and  skills 
trained  and  physical  requirements  for  interfacing 
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the  product  with  the  training  environment 

-  Obtain  necessary  information  about  the  environment 
in  which  the  user  plans  to  employ  the  product 

-  Question  current  users  of  the  product  regarding  its 
effectiveness  and  RAM  characteristics,  and  ask  users 
about  the  training  environment  in  which  it  is  used 

~  Assess  whether  the  training  device  is  compatible  with 
the  environment  in  which  it  is  intended  to  be  used, 
without  modifying  the  product  or  the  environment 

•  Conduct  a  market  survey  to  identify  other  commercial 
devices  which  might  the  requirements  at  a  lower  cost 

•  Assess  the  percentage  of  tasks  defined  in  the  training 
requirement  that  can  be  effectively  trained  with  the 
device,  without  modifying  the  device. 

•  Decide  whether  the  product  can  be  procured  on  a 
competitive  basis  versus  sole  source 

Task;  Develop  Rough  Order  of  Magnitude  (ROM)  Estimates  for 
the  Life  Cycle  Costs  of  Training  Device  Alternatives.  ROM  cost 
estimates  are  needed  to  compare  the  cost-effectiveness  of  device 
alternatives  and  to  alert  users  to  potential  funding  requirements. 
In  certain  cases,  the  cost  of  a  device  may  preclude  further 
consideration  of  that  device,  due  to  shortages  of  funds.  On 
occasion,  ROM  estimates  are  developed  in  the  absence  of  any  of  the 

documentation  described  above.  For  example,  a  school  might  decide 

that  it  needs  some  type  of  training  device  and  ask  PM  TRADE  for  a 

"ball  park”  estimate  of  the  cost  of  the  device  before  initiating 

the  concept  formulation  process.  The  responses  to  such  requests 
are  based  upon  the  same  subtasks  performed  for  all  ROM  estimates. 

Subtasks; 

•  Identify  similar  training  devices  and  review  Baseline 
Cost  Estimates  (BCE)  and  ROM  developmental  and  life 
cycle  cost  estimates  for  these  devices 

•  Select  general  method  to  be  used  for  developing 
Rough  Order  of  Magnitude  (ROM)  cost  estimates 
(e.g.,  costs  of  developing  similar  systems,  bottom 
up  estimates  or  cost  estimating  relationship) 

•  Conduct  a  market  survey  of  current  prices  for  developing 
or  buying  major  components  of  the  device,  considering 
volume  discounts,  etc. 

•  Obtain  cost  data  from  R  and  S  Division  regarding 
production  and  support  of  similar  devices 
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•  Obtain  estimates  of  the  cost  of  training  on  actual 
equipment 

•  Prepare  a  description  of  the  procedures  used  to  develop 
the  cost  estimates 

Task:  Prepare  Delivery  Order  for  a  Support  Service  Contract. 
The  scope  of  this  task  varies  among  projects.  For  example,  in 
cases  where  training  deficiencies  and  device  alternatives  have  not 
been  defined,  the  contract  may  include  a  task  analysis  and  media 
analysis. 

Subtasks: 

•  Identify  and  examine  SOWS  for  similar  efforts 

•  Prepare  SOW 

•  Estimate  cost  of  a  concept  formulation  study  (see  cost 
estimation  under  preparing  input  for  the  PSC/JWGl) 

•  Prepare  procurement  package 

•  Review  technical  proposal 

Task:  Develop  Information  for  PM  TRADE’S  Project  Screening 
Committee  fPSC^  and  Joint  Working  Group  (JVG)  I.  Prior 
to  committing  itself  to  a  project,  PM  TRADE  must  estimate  the 
size  of  the  project  and  compare  this  estimate  with  available 
resources.  Much  of  the  same  information  developed  for  this 
purpose  is  also  needed  before  JWG  I  is  able  to  decide  whether 
to  proceed  with  the  project. 

Subtasks: 

•  Decide  if  the  training  device  requirement  has  been 
adequately  described  (e.g.,  by  a  TDR  or  Training 
Development  Need  Statement) 

-  Make  a  list  of  issues  to  be  addressed  by  the  user 
to  clarify  the  training  requirement 

-  Obtain  an  estimate  from  the  school  regarding  how 
long  it  will  take  to  address  these  deficiencies 

•  Prepare  a  rough  order  of  magnitude  (ROM)  cost  estimate 
for  the  project 

-  Find  out  if  additional  work  required  to  adequately 
define  device  requirements  (see  immediately  above)  is 
to  be  performed  by  the  user  or  by  a  contractor 
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-  Identify  similar  past  or  ongoing  projects  and  find  out 
the  costs  of  these  projects 

•  Estimate  feasibility  of  project 

-  Review  technologies  to  insure  that  they  are  adequately 
documented  and  do  not  press  the  state  of  the  art  too  far 

-  Review  technological  applications  to  insure  the  planned 
applications  are  not  too  novel 

-  Identify  similar  past  and  ongoing  projects,  find  out 
what  problems  were  encountered  and  estimate  which  types 
of  problems  might  influence  the  current  project 

•  Estimate  how  long  it  would  take  to  complete 

the  project  and  compare  this  with  the  user's  time 

requirements 

-  Identify  similar  projects  and  find  out  how  long  it  took 
to  complete  these  projects 

-  Identify  problems  that  delayed  similar  projects,  and 
decide  which  problems  might  influence  the  current 
project 

•  Decide  whether  work  should  be  done  in-house  or  by 

contractor 

-  Define  the  tasks  to  be  performed  in  the  project 

-  Estimate  the  various  types  of  expertise  required  to 
execute  the  project,  and  review  the  expertise  of  in- 
house  personnel 

Task;  Conduct  a  Trade-Off  Determination  (TOP) .  The  purpose 
of  the  TOD  is  to  develop  information  about  the  various  training 
device  alternatives  to  be  considered  in  selecting  the  Best 
Technical  Approach  (BTA) . 

Subtasks: 

•  Define  issues  to  be  addressed  in  comparing  alternatives 

and  provide  information  pertinent  to  these  issues 

-  Review  TOD  and  TOA  documents  for  comparable  systems 
to  identify  relevant  issues 

-  Ask  for  HEL  review  of  TDR  to  identify  human  factors 
issues  relevant  to  the  various  alternatives  and 
data  requirements  for  addressing  these  issues 

C-6 


-  Discuss  alternatives  with  user  subject  matter  experts 
(SMEs)  to  identify  potential  strengths  and  weaknesses  of 
each  alternative 

-  Review  relevant  state  of  the  art  technology  and  training 
technology  data  through  literature  reviews,  attending 
capabilities  briefings  and  discussions  with  technical 
experts 

-  Consider  all  of  the  above  information  sources  in 
identifying  the  specific  trade-offs  to  be  addressed  by 
the  TOD 

-  Identify  data  sources  to  be  used  in  making  trade-offs 

•  Collect  data  for  the  TOD 

-  Extract  relevant  information  from  TODs  and  TOAs  from 
related  projects 

-  Collect  relevant  data  from  user  SMEs, 

technical  experts  and  reviews  of  training  documents 

-  Document  data  sources  used  in  conducting  TOD 

•  Validate  TOD 

-  Validate  sources  of  information  regarding  alternatives 

-  Insure  that  all  major  alternatives  and  issues  have 
been  addressed 

•  Prepare  TOD  document  and  revise  in  response  to  peer/ 

school  review 

•  Defend  TOD  before  PM  TRADE  representatives  during  MARBl 

and  revise  in  response  to  feedback 

Task;  Conduct  a  Trade-Off  Analysis  fTOA^ .  The  purpose  of  the 
TOA  is  to  analyze  the  data  collected  in  the  TOD  to  select  the 
best  training  device  alternative  or  combination  of  alternatives 
referred  to  as  the  Best  Technical  Approach  (BTA) .  The  goal  of 
the  TOA  may  be  to  identify  the  best  approach  for  a  given  cost, 
or  it  may  be  to  identify  the  most  cost-effective  approach 
regardless  of  cost.  According  to  formal  guidance,  the  TOA  is 
conducted  during  JWG  II;  however,  as  described  below  a  draft 
TOA  document  is  often  prepared  by  PM  TRADE  prior  to  the  JWG2  and 
distributed  to  participants  prior  to  the  meeting. 

Subtasks: 

•  Decide  how  the  trade-offs  will  be  conducted  to  identify  a 

Best  Technical  Approach  (BTA) 
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-  Review  TOA  and  BTA  for  comparable  systems  to  determine 
the  procedures  used  to  conduct  TOA  in  the  past 

-  Develop  a  plan  for  analyzing  TOD  data  to  select  a 
BTA  and  obtain  peer  review  of  this  plan 

•  Prepare  draft  TOA  for  dissemination  to  members  of 
JWG2 

-  Make  sure  that  procedures  used  in  conducting  TOA  are 
well  documented 

-  Revise  TOA  in  response  to  peer  and  school  review 

•  Prepare  to  defend  TOA  and  BTA  selection  during  MARB2 

-  Insure  that  all  procedures  used  to  obtain  data  have 
been  documented 

-  Insure  that  the  method  used  to  select  among  alternatives 
are  carefully  documented 

•  Revise  TOA  and  BTA  is  response  to  MARB  2  feedback 
Task;  Post  Concept  Formulation ♦ 

Subtasks; 

•  Prepare  Technical  Portions  of  RFP  for  Proof  of  Principle 
Effort,  etc. 

•  Participate  in  the  Review  of  Engineering  Change  Proposals 
(ECPs) 

Task;  Help  to  Guide  Research/Development  to  Insure  that 
Emerging  Technology  can  be  Effectively  Applied  to  Future  Concept 
Formulation/Device  Design  Efforts.  This  is  an  area  in  which  the 
missions  of  ED  and  EC  branch  overlap  with  the  mission  of  ET 
Branch. 

Subtasks; 


•  Keep  informed  with  state  of  the  art  technology,  including 
existing  and  planned  applications  of  the  technology 

-  Attend  contractor  presentations  within  NTSC 

-  Read  ARI,  Navy  and  AFHRL  Research  Reports  regarding 
effectiveness  of  training  devices  and  variables 
influencing  the  effectiveness  of  devices 

-  Keep  abreast  of  Navy  and  AFHRL  projects 

-  Read  technical  publications 
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•  Keep  informed  of  current  DOD  and  Army  policies  regarding 
training  devices  (white  papers,  Simulation  Advisory 
Group,  Training  Advisory  Group) 

•  Review  IRAD  project  descriptions  for  pertinent 
technologies,  applications 

•  Provide  input  for  the  PM  TRADE/ARI  Broad  Agency 
Announcement  (BAA) 

•  Review  BAA  proposals 

•  Pass  on  research  concerns  to  ET,  ARI,  and  HEL 

•  Serve  as  members  of  JCTG  subgroups  that  push  or 
investigate  the  state  of  the  art  for  a  particular  area  of 
endeavor 
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APPENDIX  D 


HOW  ET  BRANCH  INFLUENCES  FUTURE  TRAINING  TECHNOLOGY 

The  mission  of  ET  Branch  is  to  insure  that  emerging 
technology  can  be  effectively  applied  to  future  concept 
formulation  and  device  design  efforts.  Specific  tasks 
and  subtasks  performed  to  address  this  mission  are  described 
below. 

Task;  Keep  informed  with  state  of  the  art  technology. 
including  existing  and  Planned  applications  of  the  technology. 

Subtasks: 

•  Keep  informed  of  current  DOD  and  Army  policies  regarding 
training  devices  (white  papers.  Simulation  Advisory 
Group,  Training  Advisory  Group) 

•  Keep  informed  of  the  type  of  training  technology 
applications  and  issues  facing  PM  TRADE  by  attending 
PSC  reviews  of  proposed  training  projects 

•  Read  ARI,  Navy  and  AFHRL  Research  Reports  regarding 
effectiveness  of  training  devices  and  variables  influencing 
the  effectiveness  of  devices 

•  Attend  contractor  presentations  within  NTSC 

•  Attend  In-Process  Reviews  (IPRs) 

•  Read  IRAD  project  descriptions  for  pertinent  technologies 
and  applications 

•  Attend  professional  meetings  such  at  the  Interservice  and 
Industry  Training  System  Conference 

•  Conduct  site  visits 

Task;  Coordinate  with  other  Services. 

Subtasks: 

•  Represent  the  Army  on  Tri-service  standardization  projects 

•  Serve  as  members  of  JCTG  subgroups  that  push/ investigate 
the  state  of  the  art  for  a  particular  area  of  endeavor 

•  Keep  abreast  of  Navy  and  AFHRL  training  device  projects 
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Task;  Guide  Research  Projects  Funded  bv  PM-TRADE. 


Subtasks; 

•  Provide  input  for  an  Indefinite  Quantity  Contract  (IQC) 
with  the  Institute  for  Simulation  and  Training  (1ST) 

•  Monitor  execution  of  IQC  delivery  orders 

•  Provide  input  for  the  PM  TRADE/ARI 
Broad  Agency  Announcement  (BAA) 

•  Review  BAA  proposals 

•  Monitor  projects  funded  under  the  BAA 

Task;  Guide  Research  Funded  bv  Organizations  Other  Than  PM 
TRADE. 


Subtasks; 

•  Review  IRAD  project  descriptions 

for  pertinent  technologies,  applications 

•  Review  research  proposals  submitted  for  funding  by  the 
Florida  High  Technology  Research  Council 

•  Pass  on  research  concerns  to  ARI  and  HEL 

•  Keep  informed  with  state  of  the  art  technology,  including 
existing  and  planned  applications  of  the  technology 

Keep  abreast  of  Navy  and  AFHRL  projects 
-  Read  technical  publications 

•  Who,  What,  Where,  When  and  Why 

•  OPTEMPO  force  on  force  training  in  the  field  versus 
simulated  collective  crew  training  in  garrison 

•  Normal  MANPRINT  issues;  How  easy  is  the  device  to 
operate/maintain;  can  a  better  or  cheaper  device  be  made 
while  still  easy  to  operate 

•  Defining  the  integrated  training  systems  for  proposed 
systems  (institutional  stand  alone  training  devices,  weapon 
system  embedded  training  and  weapon  system  appended 
training) 
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•  Training  issues  are  almost  always  related  t-o  degree  of 
fidelity  provided  by  the  BTA.  The  school  wants  to  maximize 
fidelity  within  dollar  constraints,  while  we  want  to 
maximize  training  effectiveness  within  schedule  and  dollar 
constraints 

•  If  a  task  and  skills  analysis  is  available  (I  have  yet  to 
see  one  for  any  training  device  I  have  worked  on) ,  I  would, 
with  the  help  of  others,  determine  what  skills  are  involved 
in  the  various  tasks,  determine  if  motor  skills  and 
cognitive  skills  are  involved,  determine  how  the  operator 
and  maintainer  interact  with  the  equipment.  Usually  there 
is  insufficient  data  for  an  analysis  of  any  depth. 

•  Tailoring  device  design  to  training  audience,  design  of 
user/trainee  device  interface,  matching  trainee  skill 
requirements  [and]  device  performance/configuration. 

•  A  requirement/ function  versus  cost  of  this 
function/ requirement 

•  Training  effectiveness  and  realistic  requirements 

•  Tactical  engagement  simulation,  scoring,  training; 
precision  weapon  engagement  simulation,  scoring  and 
training;  After  Action  Review  requirements 

•  The  system  engineer  needs  to  have  a  context  perspective  of 
the  training  objectives (s) ,  training  audience  and  intended 
use  (operational  environment)  of  the  device  or  simulator 

•  Tasks  to  be  trained,  media  selection,  performance 
measurement 

•  Duplication  of  training,  training  effectiveness,  trade-off 
of  fidelity  required,  student  practice/freeplay 

•  The  [training  device]  requirements 

•  Embedded  training  applications,  MANPRINT  issues,  EIDS 
CBT,  collective/combined  arms  training  (networking) , 
MILES/AGES  requirements 

•  Training  by  soldiers  and  officers  for  army  maneuvers  in  a 
battle  situation 

•  All  types 
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•  The  Army  overall  and  we  as  an  organization  are  woefully 
delinquent  in  ‘'productizing”  the  instructional  elements  of 
devices  and  so  I  am  beginning  to  think  in  terms  of 
"teaching  machines”  opposed  to  training  devices. 

•  Mission  rehearsal 

When  asked  to  identity  the  significant  steps  or  decision  points  in 
arriving  at  the  BTA,  engineers  provided  the  responses 
listed  below. 

•  [My]  frame  of  reference  is  the  systems  engineering  guide 
provided  to  all  E  Division  [personnel]  in  August  of  88 

•  Trade-offs,  risks,  costs,  schedules,  army  manpower 
requirement,  health,  safety,  HFE  requirements,  environment 

•  Providing  an  approach  and  complete  training  device 
requirement  and  systems  training  plan  documents; 
determining,  with  the  proponent  TRADOC  agencies, 

the  various  trade-offs  available  to  meet  the  TDR/STRAP 
requirements;  analyzing  each  of  these  trade-offs  for  the 
optimum  alternative,  based  on  updated  fiscal  and  schedule 
changes 

•  Selection  and  coordination  of  the  BTA  is  accomplished 
through  trade-off  determination  and  TOA  which  retire 
consideration  of  cost,  risk,  and  training  effectiveness 

•  A  solid,  clean  TDR;  a  complete  task  analysis  (most 
critical) ;  evaluation  criteria  for  assessing  the  TOD 
leading  to  the  TOA 

•  Affordability,  training  effectiveness, 
training  requirements 

•  TOD  (what  are  the  viable  alternatives  from  the  developer's 
point  of  view,  what  tasks  need  to  trained  and  how,  what 
part  task  trainers  can  accomplish  this) ;  TOA  (which  TOD 
requirements  best  satisfy  the  user's  needs,  what  skills  can 
best  be  accomplished  by  what  method) ;  BTA  is  a  blend  of  TOD 
and  TOA 

•  Technically  simplest  approach  that  can  meet  at  least  80% 
of  the  requirements;  cost/training  effectiveness;  RAM 
consideration  and  MANPRIN”  user  friendliness;  growth 
potential;  overall  training  system  integration; 
instrumentation  with  standardized  programmable 
interchangeable  modules  (very  important) 


D-4 


•  List  alternatives,  apply  experience/ judgment 

•  Training  device  need  statement  is  generated:  training 
device  requirement  is  drafted;  JWGl  is  held;  TOD  is 
performed;  TOA  is  performed 

•  TOA  should  give  good  indication  of  the  BTA.  Any  media 
analyses  conducted  are  also  helpful.  TOD  includes  cost 
comparison,  risk  assessments,  etc. ,  all  of  which  are 
significant 

•  Cannot  answer  due  to  lack  of  experience 

•  TOD,  TOA,  BCE,  BTA-  a  selection  of  the  best  alternative 
that  stays  within  optimum  cost  efficiency 

•  JWGS,  TOD,  TOA,  ROMS  and  MARBS 

•  TOD,  TOA 

•  TOD,  TOA  and  BCE 

•  Evaluation,  analysis,  design,  development,  implementation 

•  Satisfy  requirements  with  the  best  for  the  Army's 
techniques 
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